Interactive testing system, method and electronic device

Abstract

The application provides an interactive test system, method and electronic equipment, the system comprises: a plurality of speakers; a control unit, the control unit is used for controlling a first speaker and a second speaker in the plurality of speakers to play test noise and test corpus respectively, the first speaker and the second speaker are determined from the plurality of speakers based on the application scene of the terminal under test; an analysis unit, the analysis unit is used for obtaining the response information of the terminal under test under the playing of the test noise and the test corpus, and determining the interactive test result based on the response information. The system, method and electronic equipment provided by the application realize the interactive test of the terminal under test covering different actual application scenes, and improve the reliability and comprehensiveness of the interactive test result of the terminal under test.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an interactive testing system, method, and electronic device. Background Technology

[0002] AI-based products require intelligent voice interaction testing during the early design and development phase or before delivery. Current testing methods primarily involve selecting a test site, measuring the distance between the device under test and the artificial mouth with a measuring tape, playing background noise through a speaker, and having the artificial mouth play test data. By adjusting the sound pressure levels of the speaker and the artificial mouth and setting the signal-to-noise ratio, voice recognition, voice wake-up, and other voice interaction tests are performed under a fixed signal-to-noise ratio.

[0003] However, existing interactive testing methods cover only a single actual application scenario of the tested terminal, and using only a single speaker cannot effectively simulate the actual application scenarios of different tested terminals. Therefore, the interactive testing results for many artificial intelligence products on the market are not ideal. Summary of the Invention

[0004] This invention provides an interactive testing system, method, and electronic device to address the shortcomings of existing technologies that cover only a single actual application scenario of the tested terminal and cannot effectively simulate the actual application scenarios of different tested terminals.

[0005] This invention provides an interactive testing system, comprising:

[0006] Multiple speakers;

[0007] A control unit is used to control the first speaker and the second speaker among the plurality of speakers to play test noise and test data respectively. The first speaker and the second speaker are determined from the plurality of speakers based on the application scenario of the terminal under test.

[0008] An analysis unit is used to acquire the response information of the terminal under test under the test noise and the test corpus playback, and to determine the interaction test result based on the response information.

[0009] The interactive testing system provided by this invention includes an analysis unit specifically used for:

[0010] The system acquires response information of the tested terminal in a moving state and / or a stationary state, and determines the interaction test results in the moving state and / or stationary state based on the response information.

[0011] The interactive testing system provided by this invention, when the tested terminal is in a mobile state, includes an analysis unit comprising:

[0012] The region identification and analysis subunit is used to obtain the movement path in the response information of the terminal under test when the test corpus contains a specified region, and to determine the interaction test result based on the movement path and the specified region.

[0013] And / or,

[0014] The sound source localization analysis subunit is used to obtain the movement path in the response information of the terminal under test when the test corpus contains an unspecified area, and to determine the interaction test result based on the movement path and the placement position of the second speaker.

[0015] The interactive testing system provided by this invention, wherein the region identification and analysis subunit is specifically used for:

[0016] If the test corpus contains a specified region, obtain the movement path in the response information of the terminal under test;

[0017] The target area of ​​the tested terminal is determined based on the movement path;

[0018] The interaction test effect is determined based on whether the target area is the same as the designated area, and based on the coverage area of ​​the movement path within the designated area.

[0019] The interactive testing system provided by the present invention further includes a displacement mechanism when the terminal under test is in a stationary state, and the second speaker is disposed on the displacement mechanism;

[0020] The control unit is also used to drive the displacement mechanism to adjust the position information between the second speaker and the tested terminal.

[0021] The interactive testing system provided by this invention includes an analysis unit comprising:

[0022] The static analysis subunit is used to acquire the response information of the terminal under test when the terminal under test and the second speaker are in different positions, and to determine the interaction test effect based on the speech recognition information and / or wake-up response information in the response information under different positions.

[0023] The interactive testing system provided by the present invention further includes a control unit used to equalize the plurality of speakers.

[0024] The interactive testing system provided by the present invention, when the noise type corresponding to the application scenario is regional noise, has multiple first speakers.

[0025] The present invention also provides an interactive testing method, comprising:

[0026] Based on the application scenario of the terminal under test, the first speaker and the second speaker are identified from multiple speakers;

[0027] Control the first speaker to play test noise, and control the second speaker to play test data;

[0028] The response information of the terminal under test is obtained under test noise and test corpus playback, and the interaction test result is determined based on the response information.

[0029] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the interactive testing method described above.

[0030] The interactive testing system, method, and electronic device provided by this invention, based on the application scenario of the terminal under test, uses a first speaker and a second speaker from multiple speakers. The control unit controls the first speaker and the second speaker to play test noise and test corpus respectively. Combined with the analysis unit, the response information of the terminal under test under the test noise and test corpus is obtained to determine the interactive test results. This realizes interactive testing of the terminal under test covering different actual application scenarios, and improves the reliability and comprehensiveness of the interactive test results of the terminal under test. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the structure of the interactive testing system provided by the present invention;

[0033] Figure 2 This is a flowchart illustrating the interactive testing method provided by the present invention;

[0034] Figure 3 This is a schematic diagram of the static interactive test structure provided by the present invention;

[0035] Figure 4 This is a schematic diagram of the structure of the dynamic interactive test provided by the present invention;

[0036] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0037] Figure label:

[0038] 110: Loudspeaker; 111: First loudspeaker; 112: Second loudspeaker;

[0039] 120: Control unit; 130: Analysis unit; 001: Computer equipment;

[0040] 002: Test table; 003: Terminal under test; 004: Adjustable table;

[0041] 005: Slide rail. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] Existing interactive testing methods cover only a limited range of real-world application scenarios for the terminals under test. Furthermore, they fail to effectively simulate the real-world application scenarios of different terminals under test, resulting in less than ideal and reliable interactive testing results.

[0044] To address the aforementioned issues, this invention proposes an interactive testing system that aligns with the actual usage scenarios of the terminal under test and can cover automated interactive testing across different application scenarios. Figure 1 This is a schematic diagram of the structure of the interactive testing system provided by the present invention, as shown below. Figure 1 As shown, the system includes:

[0045] Multiple speakers 110;

[0046] Control unit 120 is used to control the first speaker and the second speaker among the plurality of speakers 110 to play test noise and test data respectively. The first speaker and the second speaker are determined from the plurality of speakers 110 based on the application scenario of the terminal under test.

[0047] Analysis unit 130 is used to obtain the response information of the terminal under test under the test noise and the test corpus playback, and to determine the interaction test result based on the response information.

[0048] Here, the speaker 110 is used to play sound during the intelligent voice interaction test of the terminal under test, simulating various sounds encountered by the terminal under test in actual application scenarios. The sound mainly includes noise in the application environment and voice interaction between the user and the terminal under test, i.e., test noise and test corpus. In the interaction test process of this embodiment, considering that the test noise and test corpus may be played simultaneously or at different locations, multiple speakers 110 are used for interaction testing. The speaker 110 can generally be a speaker, artificial mouth, or other sound playback device.

[0049] Specifically, during the interactive test, firstly, the first and second speakers among the multiple speakers 110 are positioned according to the positions of various sounds in the actual application scenario of the terminal under test, and are used to play test noise and test corpus respectively. Next, the control unit 120 controls the first and second speakers in the test area to play the test noise and test corpus respectively for interactive testing. Finally, the analysis unit 130 obtains the response information of the terminal under test under the playback of test noise and test corpus, and determines whether the terminal under test correctly executes the instructions contained in the corresponding test corpus based on the response information, thereby determining the interactive test result.

[0050] Here, the first and second speakers among the multiple speakers 110 are determined by the application scenario of the terminal under test (DUT). Specifically, among the multiple speakers 110 pre-deployed in the test environment, the selection of which speakers 110, particularly their locations, serve as the first or second speakers, and the specific number of each speaker, is determined by the DUT in the actual application scenario. For example, if the DUT is typically used in a shopping mall, where noise is non-directional and comes from all directions, multiple speakers 110 can be evenly placed in the test area as the first speakers to play the test noise. Voice interaction with the DUT may come from any direction, so a single speaker 110 can be placed in any direction as the second speaker to play the test data. Conversely, if the DUT is typically used in a home, where noise may only come from certain fixed locations (e.g., the noise from a living room air conditioner or television), multiple speakers 110 can be placed in fixed positions as the first speakers to play the test noise. The voice used to interact with the terminal under test may originate from various typical locations in a home, such as the living room, kitchen, and bedroom. Multiple speakers 110 can be placed in these typical locations as secondary speakers to play the test data. Understandably, whether in a shopping mall or a home environment, there may be noise from all directions or from several fixed locations. When conducting interaction tests on the terminal under test, the first and second speakers in speaker 110 are determined based on the actual application scenario of the terminal under test to more realistically simulate the actual interaction process and obtain more reliable test data.

[0051] The first and second speakers are used to distinguish whether speaker 110 is used to play test noise or test data. For example, the first speaker can play test noise, while the second speaker can play test data. Typically, the performance requirements for the speaker 110 used to play test noise are not high; for example, a speaker enclosure can be used as the first speaker. The performance requirements for the speaker 110 used to play test data are relatively higher; for example, a professional voice-simulating device such as an artificial mouth can be used as the second speaker. Understandably, to ensure the effectiveness of interactive testing while conserving testing resources, the same speaker 110 can be used as either the first or second speaker; for example, an artificial mouth can be used to play both test data and test noise.

[0052] Furthermore, the control unit 120 here is used to control the first and second speakers among the multiple speakers 110 to play test noise and test data respectively during interactive testing. The control unit 120 can be a computer or other terminal with control functions. It is understood that the control unit 120 can achieve automated control of the first and second speakers, controlling the playback time, content, and various parameters of the test noise and test data played by the first and second speakers respectively, such as playback duration and loudness. Alternatively, the control unit 120 can pre-record scripts for the first and second speakers to play test noise and test data respectively. Compared to other interactive tests that require manual control of the speakers, this is more precise and efficient.

[0053] Finally, the analysis unit 130 here is used to acquire the response information of the terminal under test under test noise and test corpus playback, and determine the interaction test result based on the response information. The analysis unit 130 can be a terminal with analytical and computational capabilities, such as a computer. Acquiring the response information of the terminal under test under test noise and test corpus playback can be done by directly acquiring the response information of the terminal under test itself, by manually detecting the response information of the terminal under test, or by combining both methods. After obtaining the response information of the terminal under test, the interaction test result of the terminal under test can be determined based on the response information according to a pre-set formula. The interaction test result can be one or more of the following: recognition rate, wake-up rate, false wake-up rate, etc. Alternatively, the interaction test effect of the terminal under test can be determined based on the response information, such as the effect of region recognition test or sound source localization test.

[0054] It is understood that the control unit 120 and the analysis unit 130 may be located on the same processor or on different processors, and the embodiments of the present invention do not specifically limit this.

[0055] The system provided in this embodiment of the invention selects a first speaker and a second speaker from multiple speakers based on the application scenario of the terminal under test. The control unit controls the first speaker and the second speaker to play test noise and test corpus respectively. The system combines the response information of the terminal under test under the test noise and test corpus playback with the analysis unit to determine the interaction test results. This system realizes the interaction test of the terminal under test covering different actual application scenarios and improves the reliability and comprehensiveness of the interaction test results of the terminal under test.

[0056] Existing technologies primarily involve performing interaction tests on the terminal under test (DUT) when it is stationary. However, in real-world applications, the DUT is not always stationary, such as with robotic vacuum cleaners. Furthermore, the DUT generates noise when moving, which can affect the interaction test results. Therefore, performing interaction tests only when the DUT is stationary does not provide a comprehensive representation of the actual interaction test outcomes. To address this issue, based on any of the above embodiments, the analysis unit is specifically used for:

[0057] The system acquires response information of the tested terminal in a moving state and / or a stationary state, and determines the interaction test results in the moving state and / or stationary state based on the response information.

[0058] Specifically, the test terminal is categorized into a moving state and a stationary state based on its motion state during interactive testing. When the test terminal is in an interactive state, the analysis unit 130 can acquire response information from the test terminal in both the moving and / or stationary states, and determine the interactive test results based on this information. It is understood that the state of the same test terminal during interactive testing can be stationary, moving, or a combination of both. The terminal's state can be determined based on the actual application scenario of the test terminal or the specific interactive testing requirements. Typically, when the test terminal is in a moving state, compared to when it is stationary, the noise sources include noise generated during its own movement or operation. Therefore, interactive testing in a moving state more closely reflects the actual application scenario of the test terminal, resulting in more comprehensive scenario coverage for the interactive test and thus improving the reliability of the interactive test results.

[0059] By acquiring the response information of the terminal under test in mobile and / or stationary states, and determining the interaction test results in mobile and / or stationary states based on the response information, the interaction results of the terminal under test are made richer and more comprehensive, further improving the reliability of the interaction test results of the terminal under test.

[0060] Based on any of the above embodiments, when the tested terminal is in a mobile state, the analysis unit includes:

[0061] The region identification and analysis subunit is used to obtain the movement path in the response information of the terminal under test when the test corpus contains a specified region, and to determine the interaction test result based on the movement path and the specified region.

[0062] And / or,

[0063] The sound source localization analysis subunit is used to obtain the movement path in the response information of the terminal under test when the test corpus contains an unspecified area, and to determine the interaction test result based on the movement path and the placement position of the second speaker.

[0064] Specifically, when the terminal under test is in a moving state, the analysis unit 130 can analyze the speech recognition performance of the terminal under test, especially the region recognition performance in the commands, and / or analyze the sound source localization performance of the terminal under test. That is, the analysis unit 130 may include a region recognition analysis subunit and / or a sound source localization analysis subunit. Here, "the terminal under test is in a moving state" can refer to the terminal under test moving according to the commands in the test corpus during the interaction test after its first wake-up, or it can refer to the terminal moving according to the commands in the new test corpus during the execution of the interaction test, such as while it is moving. The difference between these two moving states in terms of noise is that the latter, compared to the former, has noise generated by its own movement or operation. It is understood that whether the terminal under test performs area identification testing only through the area identification analysis subunit, or only through the sound source localization analysis subunit, or both through the area identification analysis subunit and the sound source localization analysis subunit, during the interaction test is determined according to the actual application scenario of the terminal under test or according to the specific requirements of the interaction test of the terminal under test. This embodiment of the invention does not make specific limitations.

[0065] Here, the region identification and analysis subunit is used to obtain the movement path in the response information of the terminal under test when the test corpus contains a specified region. The interaction test result is determined based on whether the movement path of the terminal under test matches the specified region contained in the test corpus. The interaction test result is determined based on the location reached by the movement path and the specified region contained in the test corpus. For example, the interaction test result of the terminal under test can be determined by comparing the regions involved in the movement path, the area of ​​the regions, and the specified region contained in the test corpus. The test corpus here can be similar to phrases like "clean the kitchen" or "go sweep the living room," where the kitchen and living room are the specified regions.

[0066] Additionally, when the tested terminal is in motion, and the test corpus does not contain a specified area, a sound source localization test needs to be performed on the tested terminal using the sound source localization analysis subunit. Further, the sound source localization analysis subunit can obtain the movement path from the tested terminal's response information. Based on whether the location reached by the movement path matches the placement location of the second speaker used to play the test corpus, the interaction test result is determined. Specifically, the area within a certain radius of the second speaker's placement location can be considered the placement location of the second speaker; the specific value can be determined according to the product definition of the tested terminal. If the tested terminal reaches a certain range near the placement location of the second speaker, then the tested terminal has reached the placement location of the second speaker, meaning the interaction test result for this tested terminal is passed; otherwise, it is failed. Alternatively, the test can statistically analyze multiple instances of the tested terminal performing sound source localization and moving based on test corpus that does not contain a specified area, generating an interaction test result representing the sound source localization accuracy. The test corpus here can be corpus without a specified area, such as phrases like "Come here and scan it" or "My area is dirty, I need to clean it." The terminal's movement path can be obtained through information recorded by the terminal itself, or determined through manual detection, or located by other positioning terminals.

[0067] The system provided in this embodiment of the invention determines the interaction test results by obtaining the movement path in the response information of the terminal under test. It covers the interaction tests of the terminal under test in actual application scenarios, including those with and without specified regions in the test corpus. This makes the interaction test of the terminal under test more comprehensive and can more detailedly show the interaction test results of the terminal under test from the aspects of region recognition and / or sound source localization.

[0068] Based on any of the above embodiments, the region identification and analysis subunit is specifically used for:

[0069] If the test corpus contains a specified region, obtain the movement path in the response information of the terminal under test;

[0070] The target area of ​​the tested terminal is determined based on the movement path;

[0071] The interaction test effect is determined based on whether the target area is the same as the designated area, and based on the coverage area of ​​the movement path within the designated area.

[0072] Specifically, when the test corpus contains a specified region, the region identification and analysis subunit can obtain the movement path in the response information of the terminal under test, determine the target region of the terminal under test based on the movement path, and determine the interaction test effect by whether the target region is the same as the specified region and based on the coverage area of ​​the movement path in the specified region.

[0073] Here, the target area of ​​the terminal under test is determined based on the movement path. The area where the destination endpoint of the movement path recorded by the terminal under test is located can be used as the target area, or the target area can be manually delineated based on the movement path of the terminal under test. This embodiment of the invention does not make specific limitations on this.

[0074] Based on this determined interactive test effect, it can specifically reflect whether the target area of ​​the tested terminal is the same as the specified area contained in the test corpus, and whether the coverage area of ​​the tested terminal's movement path within the specified area meets the minimum coverage area requirement of the specified area contained in the test corpus. For example, when the tested terminal is a robot vacuum cleaner, the control unit 120 controls two speakers to play test noises, respectively simulating the sounds of an air conditioner and a television, and randomly controls an artificial mouth in any area to play test corpus, such as playing instructions like "clean the living room". The area recognition and analysis subunit obtains whether the robot vacuum cleaner cleans the living room and whether the coverage area of ​​the cleaned area meets the requirements. If the robot vacuum cleaner cleans the living room and the area of ​​the cleaned area meets the requirements, then the robot vacuum cleaner's interactive test result is passed; otherwise, it is failed. While the robot vacuum is cleaning, the control unit controls the artificial mouth in any area to play test data, such as "clean the bedroom." The area recognition and analysis subunit determines whether the robot vacuum has cleaned the bedroom and whether the area cleaned meets the requirements. If the robot vacuum cleans the bedroom and the area cleaned meets the requirements, the robot vacuum's interaction test result is passed; otherwise, it fails.

[0075] In existing technologies, interactive testing of the terminal under test mainly involves measuring the distance between the terminal under test and the human mouth using a measuring tape. This manual measurement method results in low testing efficiency when performing multiple interactive tests, and the data obtained from manual measurements has significant errors, leading to unreliable test results. To address this issue, based on any of the above embodiments, when the terminal under test is stationary, the interactive testing system further includes a displacement mechanism, with the second speaker mounted on the displacement mechanism.

[0076] The control unit is also used to drive the displacement mechanism to adjust the position information between the second speaker and the tested terminal.

[0077] The displacement mechanism mentioned here refers to a device used to adjust the distance between the second speaker and the terminal under test (DUT), such as an electric slide rail, which can achieve precise measurement and automated control. Specifically, during interactive testing, the DUT is stationary, and the second speaker used to play the test data is placed on the displacement mechanism. The distance between the second speaker and the DUT is adjusted by the displacement mechanism. This adjustment can be achieved by driving the displacement mechanism through a control unit. Compared to existing technologies that manually adjust and measure the distance between the second speaker and the DUT, using a high-precision and automatically controllable displacement mechanism yields more rigorous and reliable interactive test data, and the interactive testing efficiency is also higher.

[0078] Based on any of the above embodiments, when the tested terminal is in a static state, the analysis unit includes:

[0079] The static analysis subunit is used to acquire the response information of the terminal under test when the terminal under test and the second speaker are in different positions, and to determine the interaction test effect based on the speech recognition information and / or wake-up response information in the response information under different positions.

[0080] Specifically, when the tested terminal is in a static state during interactive testing, the analysis unit 130 further includes a static analysis subunit, used to acquire the response information of the tested terminal when the tested terminal and the second speaker are in different positional information, and to determine the interactive test effect based on the speech recognition information and / or wake-up response information in the response information under different positional information. It is understood that the different positional information referred to here can be different distances, different directions, or combinations of different distances and directions; this embodiment of the invention does not specifically limit this.

[0081] The corpus recognition information in the response information here refers to the information obtained by the tested terminal during the interactive test. It acquires the test corpus played by the second speaker, recognizes the test corpus, and executes corresponding response operations. The response operations in the response information can be considered as corpus recognition information. It can be understood that the closer the corpus recognition information in the tested terminal's response information is to the instructions contained in the test corpus played by the second speaker, the stronger the interactive test effect of the tested terminal. The interactive test effect can be determined by calculating the recognition rate of the tested terminal using the corpus recognition information in the response information. The higher the recognition rate, the stronger the corresponding interactive test effect. The recognition rate can be calculated using the following formula: Recognition Rate = (Number of Correct Recognitions / Total Number of Playbacks) * 100%.

[0082] Furthermore, the wake-up response information in the response information here refers to the information contained in the wake-up word in the test corpus played by the second speaker during the interaction test, and the terminal under test responds accordingly. For example, it may switch from standby mode to power-on mode, or reply to the test corpus with a voice message such as "Hello, what are your instructions?" This response can be considered as the wake-up response information. The terminal under test uses the wake-up response information in the response information to determine whether to enter an interactive state with the user. It is understandable that in actual application, the more accurately the terminal under test can recognize the wake-up word in the interactive voice and be successfully woken up, the stronger its interaction test performance. To determine the effectiveness of the interaction test, the wake-up rate of the tested terminal can be calculated using the wake-up response information in the response message. The wake-up rate can be calculated using the formula: Wake-up Rate = Number of Correct Wake-ups / Total Number of Playbacks * 100%. In practical applications, the interaction effect is also affected when the tested terminal is falsely woken up. Therefore, the false wake-up rate of the tested terminal can also be calculated using the wake-up response information. That is, by calculating the frequency of false wake-ups of the tested terminal within a certain period of time, it can be understood that the lower the false wake-up rate, the stronger the interaction test effect. The false wake-up rate can be specifically calculated using the formula: False Wake-up Rate = Number of False Wake-ups in 24h / 24h. It should be understood that 24h here is just a time unit; it can also be replaced by 12h, 2 days, or 1 week, etc., and this embodiment of the invention does not specifically limit this.

[0083] It is understandable that the interaction test effect can be determined by using the speech recognition information and / or wake-up response information of the terminal under test under different location information, such as calculating its recognition rate and / or wake-up rate. Specifically, the recognition rate of the terminal under test can be calculated using only the speech recognition information; or, the wake-up rate can be calculated using only the wake-up response information; or, both the recognition rate and the wake-up rate can be calculated using both speech recognition information and wake-up response information; or, other data that can measure the interaction test effect can be calculated using the information in the response information. This can be defined according to the actual application scenario and interaction test requirements of the terminal under test, and this embodiment of the invention does not impose specific limitations.

[0084] When conducting interactive tests on the terminal under test, test noise and test data are played through a speaker. Due to the inherent properties of each speaker, its ability to convert electrical signals of different frequencies into sound signals varies, resulting in a difference between the sound signal played and the sound signal itself. Therefore, when testing the same terminal under test, if different speakers are used, the test results will differ.

[0085] To address this issue, based on any of the above embodiments, the control unit is also used to equalize the plurality of speakers.

[0086] Here, the control unit can compensate for the insufficient frequency bands played by multiple speakers via external devices or by the control unit itself, thus ensuring that the speakers have a consistent ability to play sound signals of different frequencies. By equalizing multiple speakers, the differences between individual speakers are eliminated, and the equalized sound signal is closer to the frequency components of the original audio. The playback effect will not differ due to differences in the response capabilities of the speakers, thereby making the interactive test results of the tested terminal more accurate.

[0087] Based on any of the above embodiments, when the noise type corresponding to the application scenario is regional noise, the number of the first speakers is multiple.

[0088] Understandably, when the noise type in the application scenario is regional noise, the noise in the regional noise may come from all directions, and it is difficult to simulate noise from all directions within the area using only a single speaker. Therefore, in order to more closely resemble the noise type in the actual application scenario, when the noise type in the application scenario is regional noise, multiple first speakers are used to play the test noise.

[0089] Furthermore, when the noise type corresponding to the application scenario is point noise, the number of the first speakers can be one or more. In this case, each first speaker can serve as a playback device for point noise, and the number of first speakers selected depends on the amount of point noise in the application scenario.

[0090] Based on any of the above embodiments Figure 2 This is a flowchart illustrating the interactive testing method provided by the present invention, as shown below. Figure 2 As shown, the interaction testing methods include:

[0091] Step 210: Based on the application scenario of the terminal under test, determine the first speaker and the second speaker from multiple speakers;

[0092] Step 220: Control the first speaker to play test noise and control the second speaker to play test data;

[0093] Step 230: Obtain the response information of the terminal under test under test noise and test corpus playback, and determine the interaction test result based on the response information.

[0094] Specifically, based on the application scenario of the terminal under test, a first speaker and a second speaker are determined from among the multiple speakers used for interactive testing. The first speaker is designated for playing test noise, and the second speaker is used for playing test corpora. The first speaker can be determined based on the source of the noise in actual application, such as whether the noise comes from one or more specific directions, or from all directions. Alternatively, the second speaker can be determined based on the relative position of the terminal under test and the user in actual application, such as the user being 5 meters away from the terminal under test, or the user being at the same height as the terminal under test. The speaker here can be a speaker, an artificial mouth, or other sound playback device. Typically, the performance requirements for the speaker used to play test noise are not high, and a speaker can be used as the first speaker. The performance requirements for the speaker used to play test corpora are relatively higher, and a professional sound playback device that simulates human voice, such as an artificial mouth, can be used as the second speaker. Understandably, to ensure the effectiveness of interactive testing while saving testing resources, the same speaker can be used as either the first or second speaker; for example, an artificial mouth can be used to play both test corpora and test noise.

[0095] After determining the multiple speakers in the interactive test scenario, the first speaker is controlled to play test noise, and the second speaker is controlled to play test corpus. This control of the first speaker for playing test noise and the second speaker for playing test corpus can be achieved using a computer or other terminals with control functions. It is understandable that automating the control of multiple speakers allows for control over the playback time, content, and various parameters of the first and second speakers, such as playback duration and loudness. Alternatively, scripts for the first and second speakers to play test noise and test corpus can be pre-recorded by the control unit. Compared to other interactive tests that require manual speaker control, this approach is more precise and efficient.

[0096] During the interactive test, the first and second speakers of multiple speakers are used to play test noise and test data, respectively. The response information of the terminal under test (DUT) under the test noise and test data playback is then acquired, and the interactive test result is determined based on this response information. Acquiring the response information of the DUT under the test noise and test data playback can be achieved using a computer or other terminal with analytical computing capabilities. Specifically, it can be obtained by directly acquiring the response information of the DUT itself, by manually detecting the response information, or by a combination of both methods. After obtaining the response information of the DUT, the interactive test result can be determined based on a pre-set formula. The interactive test result can be one or more of the following: the DUT's recognition rate, wake-up rate, and false wake-up rate. It can be determined whether these exceed a pre-set threshold. If they exceed the threshold, the interactive test result is considered passed; otherwise, it is considered failed. Alternatively, the movement path of the DUT can be used to determine a specified area of ​​the DUT's movement path and / or the coverage area within that specified area, thereby determining the interactive test result. For example, if the specified area of ​​the movement path of the terminal under test matches the specified area contained in the test corpus, and / or the coverage area of ​​the specified area meets the minimum coverage area of ​​the specified area in the test corpus, then the interaction test result of the terminal under test is passed.

[0097] Based on any of the above embodiments Figure 3 This is a schematic diagram of the static interactive test structure provided by the present invention, as shown below. Figure 3 As shown, 111 is the first speaker, placed on a tripod, used to play test noise; 112 is the second speaker, used to play test data; 001 is a computer device, used to support the control unit 120 and the analysis unit 130 to realize the functions of both; 002 is a test table, used to place the computer device 001; 003 is the terminal under test; 004 is a height-adjustable table, used to place the terminal under test 003 and adjust the height of the terminal under test 003; 005 is a slide rail, used to adjust the distance between the terminal under test 003 and the second speaker 112.

[0098] When the terminal under test 003 is stationary in the application scenario, the test area for the terminal under test 003 can be arranged according to the following steps:

[0099] First, several first speakers 111 can be placed on height-adjustable tripods. These tripods can also be other support and adjustment devices. Adjust the tripod height to reduce sound reflection from the ground in the test area, which could interfere with the interactive test results; the first speakers 111 should be at least 1.5 meters above the ground. After fixing the first speakers 111 to the tripods, place them evenly in the test area. Multiple first speakers 111 can play test noise simultaneously to simulate noise from all directions in a shopping mall application scenario for the terminal under test (DUT). Alternatively, a single first speaker 111 can play test noise to simulate noise from a specific point in a real-world application scenario for the DUT. Generally, the ratio of the interactive test area to the maximum surface area of ​​the DUT is approximately 50:1. To maximize the testing of the interactive effect in the test area, the horizontal area of ​​the test area should be no less than 50 times the maximum surface area of ​​the DUT.

[0100] Next, place the second speaker 112 on a tripod, which is then placed on the slide rail 005. Depending on the actual application scenario of the terminal under test 003, for example, adjust the height-adjustable table 004 so that the terminal under test 003 and the second speaker 112 are at the same height, and the terminal under test 003 is directly facing the second speaker 112. The distance and direction between the second speaker 112 and the terminal under test 003 can be adjusted using the computer device 001 via the slide rail 005. The second speaker 112 is used to play test data, such as wake words and sound source location words.

[0101] Next, computer device 001 equalizes the first speaker 111 and the second speaker 112 used to play the test noise and test data, respectively. It compensates for frequency bands where the first speaker 111 and the second speaker 112 have insufficient playback capability, ensuring consistent playback capability for sound signals of different frequencies. This eliminates individual differences between the first speaker 111 and the second speaker 112. The equalized sound signal is closer to the frequency components of the original audio, and the playback effect is not affected by the different response capabilities of the first speaker 111 and the second speaker 112, making the interactive test results more accurate.

[0102] Then, set up an interactive test scenario. Based on the actual application scenario of the terminal under test 003, set the distance between the terminal under test 003 and the second speaker 112, background noise, loudness of the test corpus, and other characteristics.

[0103] Finally, the interaction test results are processed. By detecting the response information of the tested terminal 003, the success rate of the interaction test is calculated, such as the recognition rate, wake-up rate, and false wake-up rate, and the interaction test results are determined.

[0104] In interactive testing, when the tested terminal is in a mobile state, Figure 4 This is a schematic diagram of the structure of the dynamic interactive test provided by the present invention, as shown below. Figure 4 As shown, 111 is the first speaker used to play test noise; 112 is the second speaker used to play test data; 001 is a computer device used to carry the control unit 120 and the analysis unit 130 to realize the functions of both; 002 is a test table used to place the computer device 001; and 003 is the terminal under test.

[0105] When the terminal under test 003 is in a mobile state in the application scenario, the test area for the terminal under test 003 can be arranged according to the following steps:

[0106] First, multiple second speakers 112, used to simulate human interaction with the tested terminal 003, are placed in typical locations within the test area, such as bedrooms, living rooms, and kitchens—locations where voice interaction is likely to occur daily. The number of second speakers 112 is determined by the area of ​​the test area; typically, 150m² is used. 2 It is recommended to use 8 units per square meter, with an additional 10 square meters for every additional 10 square meters of area. 2 The number of second speakers 112 is increased by two. To reduce sound reflection from the ground in the test area, which could interfere with the interactive test results, and to ensure that the sound is evenly distributed throughout the rooms, the second speakers 112 in each room can be placed at a height of more than 1.5m above the ground and more than 1.2m above the walls. Then, the first speakers 111 are placed according to the location of the noise in the actual application scenario of the tested terminal 003. For example, if the tested terminal 003 is a robot vacuum cleaner in a home appliance, two first speakers 111 can be placed in the living room to play noise to simulate the noise of a television and air conditioner.

[0107] Next, computer device 001 equalizes the first speaker 111 and the second speaker 112 used to play the test noise and test data, respectively. It compensates for frequency bands where the first speaker 111 and the second speaker 112 have insufficient playback capability, ensuring consistent playback capability for sound signals of different frequencies. This eliminates individual differences between the first speaker 111 and the second speaker 112. The equalized sound signal is closer to the frequency components of the original audio, and the playback effect is not affected by the different response capabilities of the first speaker 111 and the second speaker 112, making the interactive test results more accurate.

[0108] Next, interactive tests of region recognition can be conducted. For example, test data such as "clean the living room" can be played randomly through the second speaker 112 at any location to check whether the tested terminal 003 accurately cleans the living room and to determine the area of ​​the cleaned area. Alternatively, while the tested terminal 003 is performing its work, the second speaker 112 at any location can randomly play "clean the bedroom" to check whether the tested terminal 003 responds accurately and correctly turns to execute the instructions contained in the new test data. Similarly, the correctness of the execution area and the size of the execution area are checked. The interactive test results can be determined by calculating the region recognition accuracy rate after each time the tested terminal 003 receives test data. The region recognition accuracy rate can be calculated using the following formula: Region Recognition Accuracy Rate = Number of times the region was correctly recognized / Total number of times the data was played * 100%.

[0109] Alternatively, interactive tests can be conducted to determine the sound source localization location. For example, in an interactive test scenario, the second speaker 112 in any area plays test material similar to "Come here," and the test terminal 003 is checked to see if it can move to the location where the test material is played in a timely and accurate manner. For instance, an area with a radius of 30 centimeters centered on the second speaker 112 playing the test material is considered the sound source location. If the test terminal 003 reaches this area, the sound source localization is considered correct. After the test terminal 003 reaches the sound source location, the second speaker 112 in any other area can play the test material "Come here" again, and the test terminal 003 is checked to see if it can move to the new location where the test material is played in a timely and accurate manner. The check can be performed by directly obtaining the coordinate information of the test terminal 003 itself, or by manually obtaining the location information of the test terminal. If the location information of the test terminal matches the target area in the test material, the sound source localization test result is passed; otherwise, the sound source localization test result is failed.

[0110] It is understood that when the tested terminal 003 is in a mobile state, it is subjected to area identification test and / or sound source localization test. The specific test is determined according to the actual application scenario of the tested terminal or the specific requirements of the interaction test. This embodiment of the invention does not make specific limitations.

[0111] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an interactive testing method. This method includes: determining a first speaker and a second speaker from multiple speakers based on the application scenario of the terminal under test; controlling the first speaker to play test noise and controlling the second speaker to play test corpus; acquiring the response information of the terminal under test under the test noise and the test corpus playback, and determining the interactive test result based on the response information.

[0112] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the interactive testing method provided by the above methods. The method includes: determining a first speaker and a second speaker from a plurality of speakers based on the application scenario of the terminal under test; controlling the first speaker to play test noise and controlling the second speaker to play test corpus; acquiring response information of the terminal under test under the test noise and the test corpus playback, and determining the interactive test result based on the response information.

[0114] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the interactive testing method provided by the above methods. The method includes: determining a first speaker and a second speaker from a plurality of speakers based on the application scenario of the terminal under test; controlling the first speaker to play test noise and controlling the second speaker to play test corpus; acquiring response information of the terminal under test under the test noise and the test corpus playback, and determining the interactive test result based on the response information.

[0115] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An interactive testing system, characterized in that, include: Multiple speakers; The control unit is used to control the first speaker and the second speaker among the plurality of speakers to play test noise and test data respectively. The first speaker and the second speaker are determined from the plurality of speakers based on the application scenario of the terminal under test. The first speaker and the second speaker are arranged according to the position of various sounds in the application scenario of the terminal under test. An analysis unit is configured to acquire the response information of the terminal under test under the test noise and the test corpus playback, and determine the interaction test result based on the response information; The analysis unit is specifically used for: Obtain response information of the tested terminal in a mobile state, and determine the interaction test result in the mobile state based on the response information; When the tested terminal is in a mobile state, the analysis unit includes a region identification and analysis subunit. The region identification and analysis subunit is used to obtain the movement path in the response information of the tested terminal when the test corpus contains a specified region, determine the target region of the tested terminal based on the movement path, and determine the interaction test result based on whether the target region is the same as the specified region and the coverage area of ​​the movement path in the specified region.

2. The interactive testing system according to claim 1, characterized in that, The analysis unit is also used for: Obtain the response information of the terminal under test when it is stationary, and determine the interaction test result in the stationary state based on the response information.

3. The interactive testing system according to claim 1, characterized in that, When the tested terminal is in a moving state, the analysis unit further includes: The sound source localization analysis subunit is used to obtain the movement path in the response information of the terminal under test when the test corpus contains an unspecified area, and to determine the interaction test result based on the movement path and the placement position of the second speaker.

4. The interactive testing system according to claim 2, characterized in that, When the terminal under test is stationary, the interactive testing system further includes a displacement mechanism, and the second speaker is disposed on the displacement mechanism; The control unit is also used to drive the displacement mechanism to adjust the position information between the second speaker and the tested terminal.

5. The interactive testing system according to claim 4, characterized in that, The analysis unit includes: The static analysis subunit is used to acquire the response information of the terminal under test when the terminal under test and the second speaker are in different positions, and to determine the interaction test result based on the speech recognition information and / or wake-up response information in the response information under different positions.

6. The interactive testing system according to any one of claims 1 to 5, characterized in that, The control unit is also used to equalize the plurality of speakers.

7. The interactive testing system according to any one of claims 1 to 5, characterized in that, When the noise type corresponding to the application scenario is regional noise, the number of the first speakers is multiple.

8. An interactive testing method, characterized in that, include: Based on the application scenario of the terminal under test, a first speaker and a second speaker are determined from multiple speakers. The first speaker and the second speaker are arranged according to the positions of various sounds in the application scenario of the terminal under test. Control the first speaker to play test noise, and control the second speaker to play test data; The response information of the terminal under test is obtained under test noise and test corpus playback, and the interaction test result is determined based on the response information; The step of acquiring the response information of the terminal under test under test noise and the test corpus playback, and determining the interaction test result based on the response information, includes: When the terminal under test is in a mobile state and the test corpus contains a specified region, the movement path in the response information of the terminal under test is obtained, the target region of the terminal under test is determined based on the movement path, and the interaction test result is determined based on whether the target region is the same as the specified region and the coverage area of ​​the movement path in the specified region.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the interactive testing method as described in claim 8.

Images