Automatic test method and system for voice recognition equipment

Abstract

The invention relates to an automatic test method for voice recognition equipment, and the method comprises the steps: collecting an effective voice segment, and extracting a voiceprint feature vector and a rhythm feature of the effective voice segment; a to-be-tested corpus text set is obtained, and the corpus text set comprises a standard instruction text and a generalization instruction text which are used for triggering equipment functions; inputting the voiceprint feature vector, the rhythm feature and the corpus text set into a pre-trained style migration speech synthesis model, and generating a test audio file retaining the tone and intonation style of the user; and applying the test audio file to the voice recognition test of the tested equipment. The problems of low voice simulation degree, low corpus generation efficiency and subjective result verification in the traditional test are solved; the difficulty and workload of manual acquisition are significantly reduced, and the test efficiency and coverage rate of voice recognition and intention understanding of the intelligent device and the objective accuracy of the result are improved.

Description

Technical Field

[0001] This application relates to the field of speech recognition testing, and in particular to an automated testing method, apparatus, system, computer equipment, and computer-readable storage medium for speech recognition devices. Background Technology

[0002] Currently, the main technical approaches for testing the voice recognition and intent understanding functions of smart devices are as follows: The first approach involves recording human speech using recording equipment. The drawback of this method is its inefficiency, reliance on manual labor, and difficulty in generating large-scale, rapid corpus files for speech testing, thus limiting test coverage.

[0003] The second approach involves using text-to-speech (TTS) technology to synthesize speech files. While this method improves the efficiency of file generation, the generated speech files often sound unnatural in terms of tone, intonation, and rhythm, showing a clear difference from genuine human natural speech. Therefore, using synthesized speech files for testing is insufficient to accurately reflect the device's recognition and understanding accuracy when facing natural user speech interactions.

[0004] Furthermore, existing technologies typically require manual or semi-automatic playback of audio files at the test execution level, and rely on manual checks of the device's recognition and intent understanding results for the voice commands. This testing model is inefficient, its accuracy is greatly affected by subjective factors, and it cannot achieve large-scale automated testing loops.

[0005] In view of the defects and deficiencies in the existing technologies, there is an urgent need in the field for a voice interaction testing system and method that can generate test files that closely resemble the characteristics of real human voices on a large scale and quickly, and can realize the full automation of the testing process. Summary of the Invention

[0006] This application provides an automated testing method, system, computer device, and computer-readable storage medium for speech recognition devices, to at least address the problem of low testing efficiency for speech recognition and intent understanding in related technologies.

[0007] In a first aspect, embodiments of this application provide an automated testing method for a speech recognition device, the method comprising: Collect valid speech segments and extract the voiceprint feature vector and prosodic features of the valid speech segments; Obtain a set of text corpus to be tested, wherein the set of text corpus includes standard instruction text and generalized instruction text for triggering device functions; The voiceprint feature vector, the prosodic features, and the corpus text set are input into a pre-trained style transfer speech synthesis model to generate a test audio file that retains the user's timbre and intonation style. The test audio file is used for speech recognition testing of the device under test.

[0008] In some embodiments, using the test audio file for on-device speech recognition testing includes: The test audio file is played to the device under test through an audio playback device, and the response data of the device under test is recorded simultaneously. The internal recognition logs of the device under test are obtained through the debugging interface, and the speech recognition text and intent understanding information are extracted. The speech recognition text and intent understanding information are compared with the corpus text set and the preset expected intent, respectively, and the test results are output based on the comparison results.

[0009] In some embodiments, the method further includes: Acquire preset environmental noise data, wherein the environmental noise data includes at least one of the following: background noise, appliance operating noise, and renovation noise; According to the preset signal-to-noise ratio parameters, the environmental noise data and the test audio file are digitally mixed to generate a test audio file with environmental interference. The audio playback device plays the test audio file containing environmental interference to the device under test and records the response data of the device under test simultaneously.

[0010] In some embodiments, acquiring valid speech segments includes: Collect users' raw voice data using smart terminal devices; The original speech data is uploaded to the cloud to instruct the cloud to perform speech activity detection and signal-to-noise ratio analysis on the original speech data, and to filter out valid speech segments that meet the preset quality threshold based on the results of the activity detection and signal-to-noise ratio analysis. The effective voice segment is obtained from the cloud using the smart terminal device.

[0011] In some embodiments, the smart terminal device is a terminal device in a smart home scenario, and the smart terminal device collects the user's interactive voice in the smart home scenario to obtain the raw voice data.

[0012] In some embodiments, obtaining the set of text corpus to be tested includes: Determine the basic instruction text and its corresponding standard intent; The basic instruction text is input into a pre-trained natural language processing model for semantic generalization processing to generate multiple generalized instruction texts that have the same standard intent as the basic instruction text but differ in vocabulary or sentence structure. The basic instruction text is combined with its corresponding multiple generalized instruction texts to construct the corpus text set.

[0013] In some embodiments, the method further includes: Extract the geographic location data associated with the effective speech segments, and determine the user's dialect category label based on the geographic location data; When generating the test audio file, the model parameters of the style transfer speech synthesis model are adjusted according to the dialect category label to instruct the style transfer model to generate a test audio file with dialect accent features that match the geographic location data.

[0014] Secondly, embodiments of this application provide an automated testing system for a speech recognition device, the system comprising: a data acquisition module, a preprocessing module, and an execution module, wherein: The acquisition module is used to acquire valid speech segments and extract the voiceprint feature vector and prosodic features of the valid speech segments. The preprocessing module is used to obtain a set of text corpus to be tested, wherein the set of text corpus includes standard instruction text and generalized instruction text for triggering device functions; The voiceprint feature vector, the prosodic features, and the corpus text set are input into a pre-trained style transfer speech synthesis model to generate a test audio file that retains the user's timbre and intonation style. The execution module is used to use the test audio file for speech recognition testing of the device under test.

[0015] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0017] Compared to related technologies, the present invention provides an automated testing method based on speech recognition devices. This method acquires effective speech data from users in multiple regions through an intelligent speech acquisition mechanism and combines style transfer speech synthesis technology to generate test audio files with realistic human voice timbre and emotional rhythm on a large scale. By establishing an automated closed loop of playback, sound pickup, log acquisition, and result comparison, it can achieve multi-dimensional and efficient detection of the device's speech recognition (ASR) accuracy and non-linear understanding (NLU) accuracy. This invention can significantly reduce the difficulty of manual speech acquisition and corpus processing, improve the realism of the corpus audio files, and thus greatly improve the testing efficiency of device speech recognition and NLU. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of an automated testing method for a speech recognition device according to an embodiment of this application; Figure 2 This is a structural block diagram of an automated testing system for a speech recognition device according to an embodiment of this application; Figure 3 This is a schematic diagram of the execution flow of an automated testing system for a speech recognition device according to an embodiment of this application; Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0020] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0021] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0022] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects have an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0023] The devices involved in this application include: a smart device terminal, a cloud platform, and an automated testing platform. Specifically, the smart device terminal can be a smart home control screen, a smart speaker, or other IoT terminal devices with voice interaction capabilities. This terminal device is responsible for collecting the user's voice interaction with the device in real-world scenarios, uploading the collected raw voice data to the cloud processing module, receiving test instructions sent by the automated testing platform, and sending back internal recognition logs and response data to the platform.

[0024] The cloud platform receives raw speech data uploaded from smart device terminals and performs preprocessing and feature extraction. Preprocessing includes Voice Activity Detection (VAD) and Signal-to-Noise Ratio (SNR) analysis to filter out valid speech segments that meet preset quality thresholds. Feature extraction is used to obtain the user's voiceprint feature vector, prosodic features, and geographic location metadata from the valid speech segments. The cloud also stores pre-trained style-transfer speech synthesis models.

[0025] Finally, the automated testing platform is used for corpus text processing, test audio generation, test process scheduling, and result verification. The platform has a logging interface (such as an ADB interface or serial port logging interface) for real-time acquisition of the internal recognition results of the device under test.

[0026] This embodiment describes in detail an automated testing method for a speech recognition device. Figure 1 This is a flowchart of an automated testing method for a speech recognition device according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: S101, Collect valid speech segments and extract the voiceprint feature vector and prosodic features of the valid speech segments; Specifically, this step includes the following sub-steps: S1 automatically collects user interaction voice data from real-world scenarios using smart devices (such as smart control screens and smart speakers). The collection process is continuously monitored to obtain a wide range of voice samples from people in different countries and regions. The smart device terminal performs preliminary noise reduction processing on the collected sound using an integrated microphone array.

[0027] S2, upload the raw voice data to the cloud processing module. The cloud processing module preprocesses the raw voice data, optionally including quality checks, which in turn include: Voice Activity Detection (VAD): A deep learning VAD model is used to determine the start and end times of valid human voices; Signal-to-Noise Ratio (SNR) Analysis: Calculates the SNR value of a speech segment. Only valid speech segments with an SNR value higher than a preset quality threshold and no significant clipping are selected. The quality threshold is set, for example, between 20dB and 25dB.

[0028] Duration filtering: The duration of a valid speech segment must meet the minimum reference audio length requirement of the style transfer model, for example, no less than 3 seconds.

[0029] S3, the cloud processing module performs deep feature extraction on effective speech segments.

[0030] The extracted features include: using a pre-trained voiceprint recognition network (e.g., based on x-vector or d-vector models) to extract unique timbre features of the user, such as calculating and extracting prosodic information such as fundamental frequency (F0) contour, speech rate, energy envelope, and stress position; and using an acoustic model to force alignment of speech to obtain phoneme sequences and speech duration voiceprint feature vectors.

[0031] In addition, the cloud simultaneously extracts geographic location metadata (such as GPS coordinates or IP address) provided by the smart device terminal, and associates and tags the voiceprint feature vector, prosodic features and geographic location metadata, storing them in the user's voice database.

[0032] This step utilizes smart terminals' VAD / SNR to filter effective speech segments, extract voiceprint prosody features, and associate them with geographic location information; it can build a high-quality multi-regional user voice profile library, providing a foundation for subsequent high-fidelity synthesis.

[0033] S102, obtain the set of text corpus to be tested, input the voiceprint feature vector, prosodic features and the set of text corpus into the pre-trained style transfer speech synthesis model, and generate a test audio file that retains the user's timbre and intonation style; Specifically, this step includes the following sub-steps: S1. Manually organize and input basic instruction texts and corresponding standard intents (e.g., "turn on the air conditioner") to construct a seed corpus.

[0034] S2, the automated testing platform, inputs basic instruction text into a pre-trained natural language processing (NLP) model for semantic generalization.

[0035] It should be noted that NLP models, such as sequence-to-sequence models based on the Transformer architecture, rewrite, replace, or structurally transform texts based on a pre-defined thesaurus and grammatical rules to generate multiple generalized instruction texts with the same standard intent but differing in vocabulary or sentence structure, thus facilitating the convenient and standardized organization of corpus texts. The platform uses the basic instruction texts and the generalized instruction texts together to form the corpus text set to be tested.

[0036] S3, the automated testing platform, matches and extracts corresponding voiceprint feature vectors and prosodic features from the user voice profile database based on the testing requirements of the corpus text set. This matching process can select features based on specific dialects, genders, or age groups, depending on the testing requirements.

[0037] S4 inputs the voiceprint feature vector, prosodic features and corpus text set into the pre-trained style transfer speech synthesis model.

[0038] Specifically, the voiceprint feature vector is used as input to the style encoder, and the text is used as the content input. The model fuses the information from both to generate high-fidelity test audio files that preserve the user's timbre and intonation style. The model may employ an end-to-end TTS architecture based on deep learning (such as VITS or Tacotron2 combined with a style encoder), and a vocoder (such as HiFi-GAN) converts the spectrogram into a waveform file, enabling rapid and large-scale generation of human-like speech files.

[0039] This step generates test corpora through NLP generalization, matches features, and then inputs them into a style transfer TTS model for audio synthesis; it achieves standardized text organization of the corpus and quickly generates large-scale test audio files that cover different regions, retain the user's timbre and intonation, and closely resemble real people.

[0040] S103, use the test audio file for the speech recognition test of the device under test.

[0041] Specifically, this step includes the following sub-steps: S1, the automated test platform controls the audio playback device to play test audio files to the device under test. Optionally, the playback device is, for example, a high-fidelity speaker that has been acoustically calibrated.

[0042] S2, an automated testing platform, ensures precise synchronization between audio playback commands and wake-up monitoring commands of the device under test through network protocols (such as TCP / IP) or hardware timing control, preventing the loss of the first word of voice commands due to timing deviations.

[0043] S3: After the device under test picks up the voice, it performs voice recognition and intent understanding, and makes corresponding response actions or internal log responses; the automated test platform synchronously records the response data of the device under test, which includes but is not limited to changes in device status, screen display content, or network request data.

[0044] S4, Validation of Multidimensional Results The automated testing platform obtains the internal recognition logs of the device under test in real time through debugging interfaces (such as ADB interface, JTAG interface, or remote API interface), and extracts the device's speech recognition text (ASR) and intent understanding slot information (NLU) from the logs.

[0045] Furthermore, the speech recognition text is compared with the original instruction text in the corpus. The comparison process uses a dynamic programming algorithm to calculate the word error rate (WER). The platform sets a WER threshold; if the calculated WER is higher than the threshold, the speech recognition for that use case is considered a failure.

[0046] In addition, the intent understanding slot information is simultaneously compared with the preset expected intent. The intent understanding slot information is structured data (such as a JSON object) output by the NLU model. The comparison process requires that the core fields and values, such as the intent name, subject, action, and key parameters, completely match the preset expected intent. Any inconsistency in key field or parameter values ​​is considered an intent understanding failure.

[0047] Finally, the test result is determined by combining the results of WER comparison and NLU slot comparison. A test case is considered passed and a pass report is generated only if speech recognition passes (WER is below the threshold) and intent understanding perfectly matches the expected intent. Otherwise, a failure report is generated, and the failure type (e.g., ASR failure or NLU failure) is recorded.

[0048] This step involves synchronously playing audio through an automated platform and obtaining the device's ASR / NLU internal identification logs in real time through a debugging interface; thus achieving a fully automated closed-loop testing process. By utilizing a dual comparison mechanism of WER and NLU slots, the objectivity and efficiency of the test results are improved.

[0049] Preferably, this embodiment further discloses an optimized testing scheme to improve test coverage and realism, specifically including: 1) Dialect and region compatibility test The automated testing platform uses extracted geolocation metadata to determine the user's dialect category label. When generating test audio files, based on the dialect category label, it retrieves the dialect-specific phoneme pronunciation rules and fundamental frequency offset parameters from a pre-defined dialect feature calibration library, adjusting the prosodic parameters and spectral shape of the style-transfer speech synthesis model. For example, for languages ​​from different countries or regions, it performs detailed adjustments to stress patterns and intonation curves to generate test audio files with more accurate dialect accent characteristics.

[0050] 2) Environmental noise immunity test Before playing the test audio file, preset environmental noise audio data is acquired. This environmental noise audio data includes at least one of the following: background noise (such as television sound), appliance operation noise (such as air conditioner sound), or sudden transient noise (such as door closing sound, sneezing sound). During digital mixing, the automated testing platform can selectively superimpose different types of noise for different testing purposes and precisely control the signal-to-noise ratio (SNR) range. For example, the SNR can be set to vary between 10dB and 30dB in 5dB steps to generate a series of composite test audios with different levels of interference, used to verify the anti-interference capability of the device under test in complex noise environments.

[0051] 3) Large-scale parallel test management The automated testing platform features multi-threaded scheduling and task allocation capabilities. It can manage and schedule multiple test units simultaneously, each consisting of an audio playback device and one or more devices under test. The platform assigns different test audio files (such as voiceprints from different users, different dialects, and different noise conditions) to different test units, enabling parallel execution of test cases. The platform maintains a central database that receives test results from parallel test units in real time and performs unified statistical analysis, such as calculating the average recognition failure rate under specific voiceprint styles or noise conditions, to facilitate subsequent product optimization and iteration.

[0052] Through the above steps, this solution filters effective speech features using VAD / SNR, generalizes the test corpus using an NLP model, and rapidly generates high-fidelity test audio on a large scale using a style transfer TTS model. It also employs an audio synchronization playback and debugging interface to obtain internal device logs, ultimately enabling the testing of speech recognition on terminal devices. This solution addresses the challenges of low speech fidelity, low corpus generation efficiency, and subjective result verification in traditional testing. It significantly reduces the difficulty and workload of manual data collection, improving the testing efficiency, coverage, and objective accuracy of speech recognition and intent understanding results for intelligent devices.

[0053] Secondly, this application also provides an automated testing system for speech recognition devices. Figure 2 This is a structural block diagram of an automated testing system for a speech recognition device according to an embodiment of this application, such as... Figure 2 As shown, the system includes: a data acquisition module 20, a preprocessing module 21, and an execution module 22, wherein: The acquisition module 20 is used to acquire valid speech segments and extract the voiceprint feature vector and prosodic features of the valid speech segments; The preprocessing module 21 is used to obtain the corpus text set to be tested, wherein the corpus text set contains standard instruction text and generalized instruction text for triggering device functions; and inputs the voiceprint feature vector, prosodic features and corpus text set into a pre-trained style transfer speech synthesis model to generate a test audio file that retains the user's timbre and intonation style. The execution module 22 is used to use the test audio file for the speech recognition test of the device under test.

[0054] also, Figure 3 This is a schematic diagram of the execution flow of an automated testing system for a speech recognition device according to an embodiment of this application, such as... Figure 3 As shown, this process begins on a test platform that already has cloned test audio files and expected corpus text. First, the audio playback device synchronously plays the test audio to the device under test. Then, the system obtains the device under test's internal recognition logs and response data in real time through a debugging interface, extracting the device's Automatic Speech Recognition (ASR) text and Non-Linguistic Understanding (NLU) slot information. Finally, the system performs a dual comparison and verification of the ASR text and NLU information with the original corpus and the preset expected intent, respectively, and outputs the test conclusion based on the comparison results.

[0055] The system addresses the challenges of low speech fidelity, low corpus generation efficiency, and subjective result verification in traditional testing. It significantly reduces the difficulty and workload of manual data collection, and improves the testing efficiency, coverage, and objective accuracy of speech recognition and intent understanding results for intelligent devices.

[0056] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operating system to run, the computer programs are executed by the processor to implement an automated testing method for a speech recognition device, and the database stores data.

[0057] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0058] Furthermore, in conjunction with the information interaction method of the large-screen terminal in the above embodiments, this application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements the automated testing method for any of the voice recognition devices in the above embodiments.

[0059] In one embodiment, a computer program product is provided, comprising a computer program, characterized in that, when executed by a processor, the computer program implements an automated testing method for any of the speech recognition devices described in the above embodiments.

[0060] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0061] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An automated testing method for a speech recognition device, characterized in that, The method includes: Collect valid speech segments and extract the voiceprint feature vector and prosodic features of the valid speech segments; Obtain a set of text corpus to be tested, wherein the set of text corpus includes standard instruction text and generalized instruction text for triggering device functions; The voiceprint feature vector, the prosodic features, and the corpus text set are input into a pre-trained style transfer speech synthesis model to generate a test audio file that retains the user's timbre and intonation style. The test audio file is used for speech recognition testing of the device under test.

2. The method according to claim 1, characterized in that, Using the test audio file for speech recognition testing on the device includes: The test audio file is played to the device under test through an audio playback device, and the response data of the device under test is recorded simultaneously. The internal recognition logs of the device under test are obtained through the debugging interface, and the speech recognition text and intent understanding information are extracted. The speech recognition text and intent understanding information are compared with the corpus text set and the preset expected intent, respectively, and the test results are output based on the comparison results.

3. The method according to claim 2, characterized in that, The method further includes: Acquire preset environmental noise data, wherein the environmental noise data includes at least one of the following: background noise, appliance operating noise, and renovation noise; According to the preset signal-to-noise ratio parameters, the environmental noise data and the test audio file are digitally mixed to generate a test audio file with environmental interference. The audio playback device plays the test audio file containing environmental interference to the device under test and records the response data of the device under test simultaneously.

4. The method according to claim 1, characterized in that, The collected valid speech segments include: Raw voice data from users in multiple different regions was collected using smart terminal devices. The original speech data is uploaded to the cloud to instruct the cloud to perform speech activity detection and signal-to-noise ratio analysis on the original speech data, and to filter out valid speech segments that meet the preset quality threshold based on the results of the activity detection and signal-to-noise ratio analysis. The effective voice segment is obtained from the cloud using the smart terminal device.

5. The method according to claim 4, characterized in that, The smart terminal device is a terminal device in a smart home scenario. The smart terminal device collects the user's interactive voice in the smart home scenario to obtain the raw voice data.

6. The method according to claim 1, characterized in that, The corpus text set to be tested includes: Determine the basic instruction text and its corresponding standard intent; The basic instruction text is input into a pre-trained natural language processing model for semantic generalization processing to generate multiple generalized instruction texts that have the same standard intent as the basic instruction text but differ in vocabulary or sentence structure. The basic instruction text is combined with its corresponding multiple generalized instruction texts to construct the corpus text set.

7. The method according to claim 1, characterized in that, The method further includes: Extract the geographic location data associated with the effective speech segments, and determine the user's dialect category label based on the geographic location data; When generating the test audio file, the model parameters of the style transfer speech synthesis model are adjusted according to the dialect category label to instruct the style transfer model to generate a test audio file with dialect accent features that match the geographic location data.

8. An automated testing system for a speech recognition device, characterized in that, The system includes: a terminal device, a cloud processing module, and an execution module, wherein: The acquisition module is used to acquire valid speech segments and extract the voiceprint feature vector and prosodic features of the valid speech segments. The preprocessing module is used to obtain a set of text corpus to be tested, wherein the set of text corpus includes standard instruction text and generalized instruction text for triggering device functions; The voiceprint feature vector, the prosodic features, and the corpus text set are input into a pre-trained style transfer speech synthesis model to generate a test audio file that retains the user's timbre and intonation style. The execution module is used to use the test audio file for speech recognition testing of the device under test.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images