Speech recognition performance test method and device, electronic equipment and storage medium

Abstract

The invention provides a voice recognition performance test method and device, electronic equipment and a storage medium, relates to the technical field of computers, and can highly restore a voice interaction scene in a vehicle use process through audio data carried by a voice recognition request in traffic playback data. Under the voice interaction scene, the performance of the voice recognition equipment is tested, and the accuracy and scene coverage rate of voice recognition performance testing can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a method, device, electronic device, and storage medium for testing speech recognition performance. Background Art

[0002] Speech recognition technology is an important branch in the field of artificial intelligence, involving many disciplines such as signal processing, computer science, linguistics, acoustics, physiology, psychology, etc., and is a key link in human-computer natural interaction technology. Currently, the work of speech recognition mainly relies on speech recognition devices.

[0003] How to test the performance of speech recognition devices is an issue of concern in the industry. Summary of the Invention

[0004] To solve the above problems in the prior art, embodiments of this application provide a method, device, electronic device, and storage medium for testing speech recognition performance, for testing the performance of speech recognition devices.

[0005] In a first aspect, embodiments of this application provide a method for testing speech recognition performance, the method including:

[0006] Obtain traffic playback data of multiple in-vehicle devices to obtain multiple sets of traffic playback data; the traffic playback data of any one of the multiple in-vehicle devices includes: audio data carried by a speech recognition request from the any one of the in-vehicle devices during a specified historical time period;

[0007] Send the multiple sets of traffic playback data to a speech recognition device;

[0008] According to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data, respectively determine the processing duration of the speech recognition device for each set of traffic playback data; wherein, the response data of each set of traffic playback data is obtained by the speech recognition device performing speech recognition on this set of traffic playback data;

[0009] Determine the performance of the speech recognition device according to the processing duration of the speech recognition device for each set of traffic playback data.

[0010] In a possible implementation manner, the sending the multiple sets of traffic playback data to the speech recognition device includes:

[0011] Allocate corresponding test threads for each set of traffic playback data in the multiple sets of traffic playback data;

[0012] Send the multiple sets of traffic playback data to the speech recognition device through the test threads respectively corresponding to the multiple sets of traffic playback data.

[0013] In a possible implementation, the traffic playback data of any one of the vehicle-mounted devices includes audio data carried by multiple voice recognition requests, and there is a time interval between the multiple voice recognition requests. Sending the multiple sets of traffic playback data to the voice recognition device includes:

[0014] For each set of traffic playback data in the multiple sets of traffic playback data, perform the following operations respectively:

[0015] According to the time interval between the multiple voice recognition requests of this set of traffic playback data, send this set of traffic playback data to the voice recognition device.

[0016] In a possible implementation, the traffic playback data of any one of the vehicle-mounted devices further includes: vehicle status data carried by voice recognition requests from any one of the vehicle-mounted devices within the specified historical time period; the response data of each set of traffic playback data includes text data returned by the voice recognition device for automatically performing voice recognition processing on the audio data in this set of traffic playback data, and instruction data returned by the voice recognition device for performing natural language understanding processing on the text data and the vehicle status data in this set of traffic playback data; determining the processing duration of the voice recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data includes:

[0017] For each set of traffic playback data in the multiple sets of traffic playback data, perform the following operations respectively:

[0018] Determine the first time when the text data corresponding to any audio data in this set of traffic playback data is received, and the second time when the instruction data corresponding to the text data corresponding to any audio data is received;

[0019] According to the first time, the second time, and the sending time of any audio data, determine the processing duration of the voice recognition device for this set of traffic playback data.

[0020] In a possible implementation, determining the processing duration of the voice recognition device for this set of traffic playback data according to the first time, the second time, and the sending time of any audio data includes:

[0021] According to the first time and the sending time, determine the automatic voice recognition processing duration of the voice recognition device for this set of traffic playback data;

[0022] According to the first time and the second time, determine the natural language understanding processing duration of the voice recognition device for this set of traffic playback data.

[0023] In a possible implementation manner, determining the performance of the voice recognition device according to the processing duration of the voice recognition device for each set of traffic playback data includes:

[0024] For each set of traffic playback data in the multiple sets of traffic playback data, perform the following operations respectively:

[0025] Take the automatic speech recognition processing duration of the voice recognition device for this set of traffic playback data as the first performance time-consuming of the automatic speech recognition processing of the voice recognition device for this set of traffic playback data;

[0026] Take the natural language understanding processing duration of the voice recognition device for this set of traffic playback data as the second performance time-consuming of the natural language understanding processing of the voice recognition device for this set of traffic playback data;

[0027] Take the sum of the first performance time-consuming and the second performance time-consuming as the total voice recognition performance time-consuming of the voice recognition device for this set of traffic playback data.

[0028] In a second aspect, an embodiment of the present application provides a voice recognition performance testing device, and the device includes:

[0029] An acquisition unit, configured to acquire traffic playback data of multiple in-vehicle devices to obtain multiple sets of traffic playback data; the traffic playback data of any one of the multiple in-vehicle devices includes: audio data carried by a voice recognition request from the any one of the in-vehicle devices within a specified historical time period;

[0030] A sending unit, configured to send the multiple sets of traffic playback data to a voice recognition device;

[0031] A testing unit, configured to respectively determine the processing duration of the voice recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data; wherein, the response data of each set of traffic playback data is obtained by the voice recognition device performing voice recognition on this set of traffic playback data;

[0032] Determine the performance of the voice recognition device according to the processing duration of the voice recognition device for each set of traffic playback data.

[0033] In a possible implementation manner, the sending unit is specifically configured to:

[0034] Allocate corresponding test threads for each set of traffic playback data in the multiple sets of traffic playback data;

[0035] Send the multiple sets of traffic playback data to the speech recognition device through the test threads respectively corresponding to the multiple sets of traffic playback data.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor. A computer program is stored on the memory and can run on the processor. When the computer program is executed by the processor, the method described in any one of the speech recognition performance testing methods in the first aspect is implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the method described in any one of the speech recognition performance testing methods in the first aspect is implemented.

[0038] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0039] A speech recognition performance testing method, device, electronic device and storage medium provided by an embodiment of the present application can highly restore the voice interaction scenario during vehicle use through the audio data carried in the speech recognition requests in the traffic playback data. In this voice interaction scenario, the performance of the speech recognition device is tested, which can effectively improve the accuracy and scenario coverage rate of the speech recognition performance testing. Description of the Drawings

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is an application scenario diagram of a speech recognition performance testing method provided by an embodiment of the present application;

[0042] Figure 2 It is a flowchart of a speech recognition performance testing method provided by an embodiment of the present application;

[0043] Figure 3 It is a structural schematic diagram of a speech recognition performance testing device provided by an embodiment of the present application;

[0044] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0045] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0046] It should be noted that the terms "including" and "having" and their variants involved in the documents of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0047] Some noun explanations related to the present application are as follows:

[0048] ASR (automatic speech recognition): Automatic speech recognition can enable a machine to automatically recognize and understand human speech through speech signal processing and pattern recognition. It can extract features from speech signals and map these features to corresponding natural language texts.

[0049] NLU (natural language understanding): Natural language understanding involves understanding and interpreting text data. It can map a given natural language text to instruction data that a machine can understand.

[0050] Speech recognition technology is an important branch in the field of artificial intelligence, involving many disciplines such as signal processing, computer science, linguistics, acoustics, physiology, psychology, etc., and is a key link in human-machine natural interaction technology. Currently, the work of speech recognition mainly relies on speech recognition devices.

[0051] Nowadays, speech recognition has shown strong technical advantages in fields such as smart home and automatic control. How to test the performance of speech recognition devices is an issue of concern in the industry.

[0052] Based on this, the embodiments of the present application provide a method, device, electronic device, and storage medium for testing speech recognition performance. The audio data carried by the speech recognition request in the traffic playback data can be used to highly restore the speech interaction scenario during vehicle use. In this speech interaction scenario, the performance of the speech recognition device can be tested, which can effectively improve the accuracy and scenario coverage rate of speech recognition performance testing.

[0053] First, some simple introductions will be made to the application scenarios applicable to the technical solutions of the embodiments of the present application. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application rather than to limit them. In specific implementations, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0054] Figure 1 It is a schematic diagram of the application scenario of the speech recognition performance test method provided by the embodiments of the present application. As Figure 1 shown, this application scenario includes a performance test device 10 and a speech recognition device 20. Among them, the performance test device 10 is connected to the speech recognition device 20 through a communication network.

[0055] In order to test the performance of the speech recognition device 20, multiple test threads can be set through the performance test device 10. Then, multiple test threads are used to establish connections with the speech recognition device 20 simultaneously, and traffic playback data is continuously sent to the speech recognition device 20. After the speech recognition device 20 receives the traffic playback data sent by the performance test device 10, it can perform speech recognition processing on the traffic playback data to obtain response data corresponding to the traffic playback data. Then, the speech recognition device 20 can return the response data corresponding to the traffic playback data to the performance test device 10. After the performance test device 10 receives the response data returned by the speech recognition device 20, it can determine the performance of the speech recognition device 20 according to the time when the traffic playback data is sent and the time when the response data is received.

[0056] To further illustrate the technical solutions provided by the embodiments of the present application, the following will be described in detail in combination with the accompanying drawings and specific implementation manners. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or non-creative labor. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application. When the method is actually processed or executed by the device, it can be executed in the order shown in the embodiments or drawings or executed concurrently.

[0057] Figure 2 shows a flowchart of a speech recognition performance test method provided by the embodiments of the present application, and this method can be completed by the Figure 1 shown performance test device. As Figure 2 shown, this speech recognition performance test method may include the following steps:

[0058] Step S201, obtain traffic playback data of multiple in-vehicle devices to obtain multiple sets of traffic playback data.

[0059] In an alternative embodiment, in order to test the performance of a voice recognition device, a performance testing device may obtain traffic playback data of multiple in-vehicle devices from a production log system.

[0060] Among them, the traffic playback data of any one of the multiple in-vehicle devices includes: audio data carried by voice recognition requests from any one of the in-vehicle devices during a specified historical time period. And, the traffic playback data of the any one of the in-vehicle devices includes multiple voice recognition requests, and there is a time interval between the multiple voice recognition requests.

[0061] In some embodiments, in order to ensure the quality of the obtained traffic playback data, the traffic playback data of in-vehicle devices that send high-frequency voice recognition requests may be selected.

[0062] For example, assuming that the specified historical time period is 8:00 - 8:10 in the morning of a certain day, then the traffic playback data of in-vehicle devices that send 4 or more voice recognition requests can be obtained during the time period from 8:00 to 8:10 in the morning of that day.

[0063] Step S202: Send multiple sets of traffic playback data to the voice recognition device.

[0064] After the performance testing device obtains multiple sets of traffic playback data, it can allocate corresponding test threads for each set of traffic playback data among the multiple sets of traffic playback data. Then, it uses the test threads corresponding to the multiple sets of traffic playback data to establish network connections with the voice recognition device simultaneously. After the test threads corresponding to the multiple sets of traffic playback data successfully establish network connections with the voice recognition device, it can send the multiple sets of traffic playback data to the voice recognition device.

[0065] Specifically, when the performance testing device sends multiple sets of traffic playback data to the voice recognition device, it can perform the following operations for each set of traffic playback data among the multiple sets of traffic playback data:

[0066] The performance testing device may send the set of traffic playback data to the voice recognition device according to the time interval between the multiple voice recognition requests in the set of traffic playback data.

[0067] Taking a set of traffic playback data as an example for illustration, assume that the audio data carried by 4 speech recognition requests in a set of traffic playback data are respectively "Close the window", "Turn on air conditioning for cooling", "Turn off air conditioning for cooling", and "Open the window". Among them, the time interval between the speech recognition request carrying "Close the window" and the speech recognition request carrying "Turn on air conditioning for cooling" is 2 seconds, the time interval between the speech recognition request carrying "Turn on air conditioning for cooling" and the speech recognition request carrying "Turn off air conditioning for cooling" is 180 seconds, and the time interval between the speech recognition request carrying "Turn off air conditioning for cooling" and the speech recognition request carrying "Open the window" is 10 seconds. Then, the above 4 speech recognition requests can be sent sequentially at time intervals of 2 seconds, 180 seconds, and 10 seconds.

[0068] Among them, the above is each set of traffic playback data in multiple sets of traffic playback data. The process of allocating corresponding test threads and sending multiple sets of traffic playback data to the speech recognition device through the test threads corresponding to each set of traffic playback data can be understood as a process of using multiple test threads to simulate the interaction between multiple virtual vehicles and the speech recognition device.

[0069] For example, assume that 100 sets of traffic playback data are selected, corresponding test threads are allocated for each set of traffic playback data in the 100 sets of traffic playback data, and 100 sets of traffic playback data are sent to the speech recognition device through the test threads corresponding to each of the 100 sets of traffic playback data. It can be considered that the performance testing device simulates the process of 100 virtual vehicles interacting with the speech recognition device through multiple test threads.

[0070] However, it should be noted that in actual operation, the selected traffic playback data includes but is not limited to 100 sets. Assume that the performance testing device wants to simulate the scenario of 1000 virtual vehicles interacting with the speech recognition device. Then, the selected traffic playback data can also be 1000 sets. This application does not make any limitation on the number of selected traffic playback data.

[0071] Step S203, respectively determine the processing duration of the speech recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in multiple sets of traffic playback data.

[0072] Since when a user interacts with a speech recognition device using an in-vehicle device during the process of using a vehicle, in addition to uploading the user's audio data, the in-vehicle device also needs to upload the status data of the vehicle during use. Therefore, in order to more realistically simulate the speech interaction scenario, the traffic playback data of any in-vehicle device can also include: the vehicle status data carried by the speech recognition requests from any in-vehicle device within a specified historical time period. For example, the vehicle status data can be the window opening state, vehicle speed state, air conditioning opening state, air conditioning temperature, etc.

[0073] After the performance testing device sends multiple sets of traffic playback data to the speech recognition device, it can wait in real time for the response data returned by the speech recognition device, and determine the processing duration of the speech recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data.

[0074] Among them, the response data of each set of traffic playback data includes the text data returned by the speech recognition device for performing automatic speech recognition processing on the audio data in this set of traffic playback data, and the instruction data returned by the speech recognition device for performing natural language understanding processing on the text data and the vehicle status data in this set of traffic playback data.

[0075] Specifically, for each set of traffic playback data in the multiple sets of traffic playback data, the performance testing device can respectively perform the following operations:

[0076] The performance testing device can determine the first time when the text data corresponding to any audio data in this set of traffic playback data is received, and the second time when the instruction data corresponding to the text data corresponding to any audio data is received.

[0077] Continue to take a set of traffic playback data where the audio data included in the above 4 speech recognition requests are respectively "close the window", "turn on air conditioning for cooling", "turn off air conditioning for cooling", and "open the window" as an example for illustration.

[0078] For this set of traffic playback data, the performance testing device can first send the audio data "close the window" to the speech recognition device. After the speech recognition device receives the audio data "close the window", it can perform ASR automatic speech recognition processing on the audio data "close the window", that is, convert the audio data "close the window" into text data and send the text data to the performance testing device. Then, after the performance testing device receives the text data corresponding to the audio data "close the window", it can use the time difference between the sending time of the audio data "close the window" and the first time when the text data corresponding to the audio data "close the window" is received as the automatic speech recognition processing duration of the speech recognition device for this set of traffic playback data.

[0079] Next, the speech recognition device can perform NLU natural language understanding processing on the text data corresponding to the audio data "close the window" in this set of traffic playback data and the vehicle status data carried in the speech recognition request in this set of traffic playback data, that is, convert the text data corresponding to the audio data "close the window" into corresponding instruction data, and send the instruction data to the performance testing device. After receiving the instruction data corresponding to the text data corresponding to the audio data "close the window", the performance testing device can use the time difference between the second moment when the instruction data is received and the first moment when the text data corresponding to the audio data "close the window" is received as the natural language understanding processing duration of the speech recognition device for this set of traffic playback data.

[0080] After the performance testing device determines the processing duration of the speech recognition device for the speech recognition request carrying the audio data "close the window" in this set of traffic playback data, it can continue to send the speech recognition request carrying other audio data in this set of traffic playback data to the speech recognition device according to a preset time interval, so as to determine the processing duration of the speech recognition device for the speech recognition request carrying other audio data in this set of traffic playback data. Among them, the process of determining the processing duration of the speech recognition request carrying other audio data in this set of traffic playback data is the same as the process of determining the processing duration of the speech recognition request carrying the audio data "close the window", and will not be elaborated here.

[0081] Step S204, determine the performance of the speech recognition device according to the processing duration of the speech recognition device for each set of traffic playback data.

[0082] After the performance testing device determines the processing duration of the speech recognition device for each set of traffic playback data, for each set of traffic playback data in multiple sets of traffic playback data, the following operations can be performed respectively:

[0083] The performance testing device can use the ASR automatic speech recognition processing duration of the speech recognition device for this set of traffic playback data as the first performance time-consuming of the ASR automatic speech recognition processing of the speech recognition device for this set of traffic playback data.

[0084] Furthermore, the NLU natural language understanding processing duration of the speech recognition device for this set of traffic playback data can be used as the second performance time-consuming of the NLU natural language understanding processing of the speech recognition device for this set of traffic playback data.

[0085] Next, the sum of the first performance time-consuming and the second performance time-consuming can be used as the total speech recognition performance time-consuming of the speech recognition device for this set of traffic playback data.

[0086] A method, device, electronic device, and storage medium for testing speech recognition performance provided by an embodiment of the present application can highly restore the speech interaction scenario during vehicle use through the audio data carried in the speech recognition requests in the traffic playback data, thereby solving the problems of lack of data resources and poor authenticity in speech recognition testing.

[0087] The traffic playback data in the present application is obtained by replaying the speech recognition requests of in-vehicle devices during a historical time period. Therefore, it can better copy and reproduce the user's speech interaction scenario, ensure the authenticity of the speech recognition test scenario, and thus improve the accuracy of speech recognition testing. Moreover, the traffic playback data obtained through traffic playback can cover various scenarios in the daily use of the vehicle, which helps to improve the scenario coverage rate of speech recognition testing.

[0088] Based on the same inventive concept, an embodiment of the present invention also provides a structural schematic diagram of a speech recognition performance testing device, as Figure 3 shown. The speech recognition performance testing device includes:

[0089] An acquisition unit 301, configured to acquire traffic playback data of multiple in-vehicle devices to obtain multiple sets of traffic playback data; the traffic playback data of any one of the multiple in-vehicle devices includes: audio data carried in the speech recognition requests from any one of the in-vehicle devices during a specified historical time period;

[0090] A sending unit 302, configured to send multiple sets of traffic playback data to a speech recognition device;

[0091] A testing unit 303, configured to respectively determine the processing duration of the speech recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data; wherein, the response data of each set of traffic playback data is obtained by the speech recognition device performing speech recognition on the set of traffic playback data;

[0092] Determine the performance of the speech recognition device according to the processing duration of the speech recognition device for each set of traffic playback data.

[0093] Based on the same inventive concept, an embodiment of the present application further provides an electronic device, which may be the performance testing device described above. The electronic device at least includes a memory for storing data and a processor. For the processor for data processing, when performing processing, it may be implemented by a microprocessor, a CPU, a GPU (Graphics Processing Unit), a DSP, or an FPGA. For the memory, operation instructions are stored in the memory, and the operation instructions may be computer-executable code, and each step in the process of the voice recognition performance testing method of the embodiment of the present application is implemented through the operation instructions.

[0094] Figure 4 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 4 shown, the electronic device 400 includes a memory 401, a processor 402, a data acquisition module 403, and a bus 404. The memory 401, the processor 402, and the data acquisition module 403 are all connected through the bus 404, and the bus 404 is used for transmitting data between the memory 401, the processor 402, and the data acquisition module 403.

[0095] Among them, the memory 401 can be used to store software programs and modules. The processor 402 executes various functional applications and data processing of the electronic device 400 by running the software programs and modules stored in the memory 401, such as the voice recognition performance testing method provided by the embodiment of the present application. The memory 401 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs of at least one application, etc.; the data storage area may store data created according to the use of the electronic device 400, etc. In addition, the memory 401 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0096] The processor 402 is the control center of the electronic device 400, connects various parts of the entire electronic device 400 through the bus 404 and various interfaces and lines, and executes various functions of the electronic device 400 and processes data by running or executing the software programs and / or modules stored in the memory 401, and calling the data stored in the memory 401. Optionally, the processor 402 may include one or more processing units, such as a CPU, a GPU (Graphics Processing Unit), a digital processing unit, etc.

[0097] The embodiments of the present application also provide a computer-readable storage medium. Computer-executable instructions are stored in the computer storage medium. When the computer program is executed by a processor, it can be used to implement the speech recognition performance testing method described in any embodiment of the present application.

[0098] In some possible implementation manners, each aspect of the speech recognition performance testing method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps of the speech recognition performance testing method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the process of the Figure 2 speech recognition performance testing method as shown.

[0099] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0101] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps for implementing the functions specified in one block or a plurality of blocks.

[0103] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations therein.

Claims

1. A method for testing speech recognition performance, characterized in that, The method includes: Obtaining traffic playback data of multiple in-vehicle devices to obtain multiple sets of traffic playback data; the traffic playback data of any one of the multiple in-vehicle devices includes: audio data carried by voice recognition requests from the any one of the in-vehicle devices during a specified historical time period; Sending the multiple sets of traffic playback data to a voice recognition device; Respectively determining the processing duration of the voice recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data; wherein, the response data of each set of traffic playback data is obtained by the voice recognition device performing voice recognition on the set of traffic playback data; Determining the performance of the voice recognition device according to the processing duration of the voice recognition device for each set of traffic playback data.

2. The method according to claim 1, characterized in that, The sending the multiple sets of traffic playback data to the voice recognition device includes: Allocating corresponding test threads for each set of traffic playback data in the multiple sets of traffic playback data; Sending the multiple sets of traffic playback data to the voice recognition device through the test threads corresponding to the multiple sets of traffic playback data respectively.

3. The method according to claim 1, wherein The traffic playback data of any one of the in-vehicle devices includes audio data carried by multiple voice recognition requests, and there is a time interval between the multiple voice recognition requests. The sending the multiple sets of traffic playback data to the voice recognition device includes: For each set of traffic playback data in the multiple sets of traffic playback data, respectively perform the following operations: Sending the set of traffic playback data to the voice recognition device according to the time interval between the multiple voice recognition requests of the set of traffic playback data.

4. The method according to claim 1, characterized in that The traffic playback data of any one of the in-vehicle devices further includes: vehicle status data carried by voice recognition requests from the any one of the in-vehicle devices during the specified historical time period; the response data of each set of traffic playback data includes text data returned by the voice recognition device performing automatic speech recognition processing on the audio data in the set of traffic playback data, and instruction data returned by the voice recognition device performing natural language understanding processing on the text data and the vehicle status data in the set of traffic playback data; the respectively determining the processing duration of the voice recognition device for each set of traffic playback data according to the reception time of the response data of each set of traffic playback data in the multiple sets of traffic playback data includes: For each set of traffic playback data in the multiple sets of traffic playback data, respectively perform the following operations: Determining a first time when the text data corresponding to any audio data in the set of traffic playback data is received, and a second time when the instruction data corresponding to the text data corresponding to the any audio data is received; Determining the processing duration of the voice recognition device for the set of traffic playback data according to the first time, the second time, and the sending time of the any audio data.

5. The method according to claim 4, characterized in that, The determining the processing duration of the voice recognition device for the set of traffic playback data according to the first time, the second time, and the sending time of the any audio data includes: Determine the automatic speech recognition processing duration of the speech recognition device for this set of traffic playback data according to the first moment and the sending moment; Determine the natural language understanding processing duration of the speech recognition device for this set of traffic playback data according to the first moment and the second moment.

6. The method according to claim 5, wherein Determining the performance of the speech recognition device according to the processing duration of the speech recognition device for each set of traffic playback data includes: For each set of traffic playback data in the multiple sets of traffic playback data, perform the following operations respectively: Use the automatic speech recognition processing duration of the speech recognition device for this set of traffic playback data as the first performance consumption time for the automatic speech recognition processing of the speech recognition device for this set of traffic playback data; Use the natural language understanding processing duration of the speech recognition device for this set of traffic playback data as the second performance consumption time for the natural language understanding processing of the speech recognition device for this set of traffic playback data; Use the sum of the first performance consumption time and the second performance consumption time as the total speech recognition performance consumption time of the speech recognition device for this set of traffic playback data.

7. A voice recognition performance testing device, characterized in that, The device includes: An acquisition unit, configured to acquire traffic playback data of multiple in-vehicle devices to obtain multiple sets of traffic playback data; the traffic playback data of any one of the multiple in-vehicle devices includes: audio data carried in a speech recognition request from the any one of the in-vehicle devices within a specified historical time period; A sending unit, configured to send the multiple sets of traffic playback data to a speech recognition device; A testing unit, configured to respectively determine the processing duration of the speech recognition device for each set of traffic playback data according to the reception moment of the response data of each set of traffic playback data in the multiple sets of traffic playback data; wherein, the response data of each set of traffic playback data is obtained by the speech recognition device performing speech recognition on this set of traffic playback data; Determine the performance of the speech recognition device according to the processing duration of the speech recognition device for each set of traffic playback data.

8. The voice recognition performance testing device according to claim 7, wherein, The sending unit is specifically configured to: Allocate corresponding test threads for each set of traffic playback data in the multiple sets of traffic playback data; Send the multiple sets of traffic playback data to the speech recognition device through the test threads corresponding to the multiple sets of traffic playback data respectively.

9. An electronic device, characterized in that, It includes a memory and a processor, and a computer program is stored on the memory and can run on the processor. When the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program therein, characterized in that: When the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images