Voice Test Method, Device, Electronic Device and Storage Medium

Abstract

The present disclosure provides a voice test method, apparatus, electronic device, and storage medium, which relate to the field of artificial intelligence technologies, and particularly to the field of natural language processing technologies. The specific implementation solution is as follows: send multiple clauses in the test voice sentence by sentence, and record the sending time parameter of each clause; receive the speech recognition result of the test voice sentence by sentence, and record the receiving time parameter of the speech processing result of each clause; determine the delay time based on the sending time parameter of each clause and the receiving time parameter of the speech processing result of each clause. In the embodiments of the present disclosure, for the speech processing of long voices, the sending time parameter and the receiving time parameter of each clause can be recorded, thereby implementing the delay time required for speech processing of long voices, filling the blank of the lack of a delay time test method in the case of long voices.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to the field of natural language processing technology. Background Art

[0002] Based on natural language processing (NLP, Natural Language Processing) technology, functions such as speech transcription and human-computer interaction have been realized.

[0003] Natural language processing cannot do without a speech recognition model. The speech recognition model is used to recognize the input speech to realize functions such as speech transcription or human-computer interaction. There may be certain differences in the speech processing capabilities of different speech recognition models. In order to measure the speech processing capabilities of the speech recognition model, it needs to be tested. The testing methods proposed in the related technologies are only applicable to speeches with short durations, and no testing scheme has been proposed for long speeches with long durations. Summary of the Invention

[0004] The present disclosure provides a speech testing method, apparatus, electronic device and storage medium

[0005] According to the first aspect of the present disclosure, a speech testing method is provided, including:

[0006] Sending multiple clauses in the test speech sentence by sentence, and recording the sending time parameter of each clause;

[0007] Receiving the speech recognition results of the test speech sentence by sentence, and recording the receiving time parameter of the speech processing result of each clause;

[0008] Determining the delay time based on the sending time parameter of each clause and the receiving time parameter of the speech processing result of each clause.

[0009] According to the second aspect of the present disclosure, a speech testing apparatus is provided, including:

[0010] A sending module, configured to send multiple clauses in the test speech sentence by sentence, and record the sending time parameter of each clause;

[0011] A receiving module, configured to receive the speech recognition results of the test speech sentence by sentence, and record the receiving time parameter of the speech processing result of each clause;

[0012] An evaluation module, configured to determine the delay time based on the sending time parameter of each clause and the receiving time parameter of the speech processing result of each clause.

[0013] According to the third aspect of the present disclosure, an electronic device is provided, including:

[0014] At least one processor; and,

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method of the foregoing first aspect.

[0017] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method of the foregoing first aspect.

[0018] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, implements the method of the foregoing first aspect.

[0019] In the embodiments of the present disclosure, for the voice processing of long voices, the transmission time parameter and the reception time parameter of each clause can be recorded, thereby implementing the delay time required for voice processing of long voices, filling the gap in the absence of a delay time test method in the case of long voices.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0022] Figure 1 is a schematic flowchart of a voice test method according to an embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram of sending a test voice and receiving a voice recognition result in a voice test method according to an embodiment of the present disclosure;

[0024] Figure 3 is another schematic flowchart of a voice test method according to an embodiment of the present disclosure;

[0025] Figure 4 is a schematic diagram of explaining the transmission time parameter and the reception time parameter for clauses according to an embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram of the annotation position of a test voice according to an embodiment of the present disclosure;

[0027] Figure 6It is a schematic diagram showing the byte positions for annotating the positions of test voices according to an embodiment of the present disclosure;

[0028] Figure 7 It is another schematic diagram showing the transmission time parameter and the reception time parameter according to an embodiment of the present disclosure;

[0029] Figure 8 It is a schematic diagram showing the end reception time according to an embodiment of the present disclosure;

[0030] Figure 9 It is another flowchart schematic diagram of the voice test method according to an embodiment of the present disclosure;

[0031] Figure 10 It is a schematic diagram showing the clauses to be discarded according to an embodiment of the present disclosure;

[0032] Figure 11 It is a schematic diagram of the architecture of the voice test product according to an embodiment of the present disclosure;

[0033] Figure 12 It is a schematic diagram showing the determination of the response time and the delay according to an embodiment of the present disclosure;

[0034] Figure 13 It is another schematic diagram showing the reception time parameter and the transmission time parameter according to an embodiment of the present disclosure;

[0035] Figure 14 It is a schematic diagram of the structure of a voice test device according to an embodiment of the present disclosure;

[0036] Figure 15 It is another schematic diagram of the structure of a voice test device according to an embodiment of the present disclosure;

[0037] Figure 16 It is a block diagram of an electronic device for implementing the voice test method of the embodiments of the present disclosure. Detailed implementation manners

[0038] The following makes an explanation of the exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.

[0039] The terms "first", "second", etc. in this disclosure are used to distinguish similar objects and do not necessarily describe a specific order or sequence. In addition, the terms "comprising", "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a method, system, product or device that comprises a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0040] Short speech can be understood as speech with a short audio duration, and long speech can be understood as speech with a long audio duration. In one possible implementation, speech with an audio duration exceeding a duration threshold can be called long speech, and speech with an audio duration not exceeding the duration threshold can be called short speech. The duration threshold can be set to 60s (seconds), for example, and the specific setting of the duration threshold can be flexibly determined according to business requirements.

[0041] For the purpose of testing the processing ability of a speech recognition model for long speech, an embodiment of this disclosure provides a speech testing method, as Figure 1 shown in the flowchart of the method. The method includes the following steps:

[0042] S101, send multiple clauses in the test speech sentence by sentence and record the sending time parameters of each clause.

[0043] During implementation, a sending queue is used to send each clause in the test speech sentence by sentence. As Figure 2 shown, when sending the test speech, a parameter packet is sent first, the test speech is divided into multiple audio packets, and sending each packet in order is equivalent to sending the test speech sentence by sentence. The size (i.e., the amount of data) of each audio packet is the same, and the content of an audio packet sent in the middle may be the content of one clause or the content of two adjacent clauses.

[0044] When sending each packet, the sending time parameters of each clause can be recorded. Among them, the sending time parameters can include the start sending time and / or the end sending time of each clause.

[0045] Correspondingly, as Figure 2 shown, after receiving the audio packet, the speech recognition model obtains the audio data of the test speech, detects each clause from the received audio data and performs recognition to obtain the speech recognition result. The operation of receiving speech recognition is as in step S102.

[0046] S102, receive the speech recognition results of the test speech sentence by sentence and record the receiving time parameters of the speech processing results of each clause.

[0047] The method of receiving the speech recognition result is similar to the method of sending the test speech and can be received using a receiving queue. The returned speech recognition results are also sent packet by packet, and the receiving time parameter of the speech recognition result of each clause can be recorded. The receiving time parameter includes the start receiving time and / or the end receiving time.

[0048] S103. Determine the delay time based on the sending time parameter of each clause and the receiving time parameter of the speech processing result of each clause.

[0049] Among them, the delay time is used to measure the time-consuming required for speech processing. The time-consuming indicator may include the response time. The response time can be understood as the duration from when the first word of a clause is sent until the start of receiving the speech recognition result of that clause. This can well measure the response delay of each clause in a long speech.

[0050] In the embodiments of the present disclosure, for the test speech of a long speech, the sending time parameter and the receiving time parameter of each clause can be recorded. Thus, it is beneficial to determine the delay time when performing speech processing on the long speech from the clause granularity based on the sending and receiving situations of each clause and the speech recognition result, filling the blank of the delay time test for long speeches.

[0051] As described above, in the embodiments of the present disclosure, the sending time parameter includes the start sending time, the receiving time parameter includes the start receiving time, and the delay time includes the response time. Based on this, determining the response time can be implemented as the following steps, as Figure 3 shown including:

[0052] S301. For each clause, determine the first time difference between the start sending time of the clause and the start receiving time of the speech recognition result of the clause;

[0053] S302. Perform statistical analysis on the first time differences corresponding to each clause respectively to obtain the response time.

[0054] As Figure 4 shown, for any clause query-A in the test speech, the start sending time rst1 of the clause is recorded, and the start receiving time rst1' of the speech recognition result of the clause is recorded. Thus, the first time difference for the clause query-A is (rst1' - rst1).

[0055] After obtaining the first time differences of each clause in the test speech, the mean value of the first time differences of each clause can be statistically calculated as the test result of the response time.

[0056] Of course, in some other embodiments, the maximum and minimum values of the first time differences of each clause can also be used as the test results of the response time. In specific implementation, the statistical analysis method for the first time difference can be determined according to actual requirements, and the embodiments of the present disclosure do not limit this.

[0057] Since the test voice is sent packet by packet, and the speech recognition results are also received packet by packet, the time delay between the start sending time and the start receiving time calculated can also be referred to as the first-packet response time.

[0058] In the embodiments of the present disclosure, based on the start sending time of each clause and the start receiving time of the speech recognition results of each clause, the response time required for speech processing can be accurately measured.

[0059] In addition to the response time described above, the test metrics for long speech in the embodiments of the present disclosure can also include hard delay. For each clause, the hard delay refers to the duration from the end of the last word of the clause to the completion of receiving the speech recognition result of the clause. Based on this, the sending time parameter includes the end sending time, the receiving time parameter includes the end receiving time, and the delay time includes the hard delay. Then the method for determining the hard delay can be implemented as Figure 3 shown, including the following steps:

[0060] S303: For each clause, determine the second time difference between the end sending time of the clause and the end receiving time of the speech recognition result of the clause.

[0061] S304: Conduct statistical analysis on the second time differences corresponding to each clause respectively to obtain the hard delay for speech processing of the test voice.

[0062] Continuing with Figure 4 as an example, for any clause query-A in the test voice, the end sending time ret1 of the clause is recorded, and the end receiving time ret1' of the speech recognition result of the clause is recorded. Thus, the second time difference for clause query-A is (ret1' - ret1).

[0063] After obtaining the second time differences of each clause in the test voice, the mean value of the second time differences of each clause can be statistically obtained as the hard delay for speech processing of the test voice.

[0064] Of course, in some other embodiments, the maximum and minimum values of the second time differences of each clause can also be used as the hard delay. In specific implementation, the statistical analysis method for the second time difference can be determined according to actual requirements, and the embodiments of the present disclosure do not limit this.

[0065] In the embodiments of the present disclosure, based on the end sending time of each clause and the end receiving time of the speech recognition result of each clause, the hard delay index for processing long speech can be accurately measured.

[0066] In the embodiments of the present disclosure, in order to facilitate accurately recording the start sending time and end sending time of each clause in the test speech, each clause in the test speech is marked in a marked manner. Each clause in the test speech has a marked position, as Figure 5 shown, the marked position may include the start position and end position of each clause in the test speech. When the response time needs to be tested, only the start position is used for marking. When the hard delay needs to be tested, only the end position is used for marking. Thus, in the embodiments of the present disclosure, recording the sending time parameter of each clause can be implemented as: obtaining the marked position of each clause in the test speech; based on the marked position of each clause, recording the sending time parameter of each clause.

[0067] In the embodiments of the present disclosure, based on the marked position of the test speech, the position of each clause in the test speech can be accurately located, so as to provide strong data support for accurately recording the sending time parameter and receiving time parameter of each clause.

[0068] Next, taking the test of both the response time and the hard delay as an example, the speech test method of the embodiments of the present disclosure will be further described.

[0069] When recording the sending time parameter, the start sending time in the sending time parameter of each clause can be recorded based on the start position in the marked position of each clause; the end sending time in the sending time parameter of each clause can be recorded based on the end position in the marked position of each clause.

[0070] Thus, in the embodiments of the present disclosure, based on the start position of the clause in the test speech, the start sending time of the clause can be accurately recorded, and based on the end position of the clause in the test speech, the end sending time of the clause can be accurately recorded. Accurately recording the start sending time and end sending time of each clause is beneficial to improving the accuracy of the test.

[0071] For ease of understanding, the following further describes how to record the start sending time and end sending time.

[0072] Figure 5 The marked position shown in is based on the time position of the test speech. When sending, in order to facilitate determining when each clause starts sending and when it ends sending, in the present disclosure, the start position and end position of each clause in the marked position are converted into byte positions.

[0073] Based on Figure 2As can be seen from the content, the test voice is sent packet by packet from the sending queue. Assuming that the size of each packet is α bytes and one packet is sent every time interval T, for any clause, the calculation method for converting its start position or end position into a byte position is as shown in formula (1):

[0074]

[0075] In formula (1), w represents the byte position, t represents the marked position of any clause in the test voice. When t is the start position, w is the byte position of the start position; when t is the end position, w is the byte position of the end position. T represents the sending time interval of each packet (for example, 160 ms), and α represents the size of each packet.

[0076] Thus, after obtaining the marked positions of each clause in the test voice, the marked positions can be converted into byte positions that are easy to identify during sending based on formula (1). As shown in Table 1, it is an example after converting the marked positions of each clause in the test voice into byte positions.

[0077] Table 1

[0078] Sub - clause Byte position at the start position Byte position at the end position Sub - clause 1 Sb1 Eb1 Sub - clause 2 Sb2 Eb2 …… …… …… Sub - clause n Sbn Ebn

[0079] With the data shown in Table 1, as Figure 6 shown, starting from the first audio byte of the test voice, assuming the byte position of the start position of the first clause is 2,656,000, it means that when sending the 2,656,000th byte, the first clause starts to be sent. Assuming the byte position of the end position of the first clause is 2,944,000, it means that when sending the 2,944,000th byte, the first clause finishes being sent.

[0080] Similarly, assuming the byte position of the start position of the second clause is 3,500,000, it means that when sending the 3,500,000th byte, the second clause starts to be sent, and so on. The byte position at which each clause starts to be sent and the byte position at which it finishes being sent can be determined. Thus, it can be known that the byte position is essentially equivalent to the number of bytes sent. Therefore, when sending the test voice, the number of bytes sent can be counted. When the counted number of bytes sent reaches the byte position of the start position of the first clause, time is marked to obtain the start sending time of the first clause. When sending reaches the byte position of the end position of the first clause, time is marked to obtain the end sending time of the first clause, and so on. The start sending time and end sending time of each clause in the test voice can be obtained.

[0081] Since the test voice is sent packet by packet, in order to facilitate the statistics of the number of bytes sent, in the embodiments of the present disclosure, the number of bytes sent is accumulated once for each packet sent, and the accumulated value is the size of one packet. For example, after sending the first packet, the number of bytes sent is a1. Compare a1 with the byte position a of the start position of the first clause. If a1 < a, it means that the first clause has not started to be sent yet. Continue to send the second packet and accumulate a1. The accumulated quantity is the number of bytes of the second packet, and a1 is updated accordingly. After accumulation, continue to judge the size of a1 and a until a1 > a and all the accumulated sent packets have been sent, which means that the first clause has started to be sent. At this time, mark the time to get "rst1". Since the last byte of a sent packet may not be the last byte of the first clause, in order to obtain the accurate start time of the first clause, the offset between the current a1 and a can be further calculated, that is, the difference a' between the two. Then, correct "rst1" according to the offset a' to obtain the accurate start time of the first clause. For any clause, the method of correcting the sending time parameter is shown in formula (2):

[0082]

[0083] In formula (2), rsti represents the start sending time of the i-th clause, rsti" represents the marked time of the start sending time of the i-th clause, ai' represents the offset between the accumulated number of bytes sent and the number of bytes at the start position of the i-th clause, T represents the sending time interval of each packet (for example, 160 ms), and α represents the size of the sent packet.

[0084] Similarly, for the marked positions of each clause, the start sending time and end sending time of each clause can be calculated in the above manner. For the end sending time, the parameters in formula (2) can be understood as: rsti represents the end sending time of the i-th clause, rsti" represents the marked time of the end sending time of the i-th clause, ai' represents the offset between the accumulated number of bytes sent and the number of bytes at the end position of the i-th clause, T represents the sending time interval of each packet (for example, 160 ms), and α represents the size of the sent packet.

[0085] During the process of sending the test voice to the speech recognition model for speech processing, voice activity detection (VAD) needs to be performed on the test voice. Thus, each clause (hereinafter referred to as the detected clause) is segmented from the received test voice through VAD. Then, speech recognition is performed on the detected clause, and the speech recognition result is sent back afterwards.

[0086] Correspondingly, the returned speech recognition results are also received packet by packet. In the returned speech recognition results, the received packets of the same detected clause can be identified based on the clause identification in the received packets. For example, the first packet of a detected clause carries a first specified symbol, and the last packet carries a second specified symbol. Thus, when receiving the returned packets in a streaming manner, the received packets of each detected clause can be identified based on the clause identification. For each clause, when the first received packet of the clause is received, time is marked to obtain the start receiving time of the speech recognition result of the clause. When the last received packet of the clause is received, time is marked to obtain the end receiving time of the clause.

[0087] When using VAD for clause segmentation, the obtained clause results may be inconsistent with the annotated clause results. This inconsistency is reflected in that the number of detected clauses may be inconsistent with the number of annotated clauses, or there may be differences in the annotated position of the same clause in the test speech and the start and end positions detected by VAD. During implementation, it is necessary to align the annotated clauses and the detected clauses to ensure that the time parameters (including the sending time parameters and receiving time parameters) of the same clause are used for testing when calculating the response time and hard delay, so as to improve the accuracy of the delay time test.

[0088] In a possible implementation manner, it can be determined whether the number of detected clauses is the same as the number of clauses annotated in the test speech. If not, it indicates that there is an error in the VAD of the speech recognition model.

[0089] Even if the number of clauses is the same, there may be a situation where the segmentation positions of the same clause are different. For example Figure 7 As shown, when using VAD to segment the test speech into clauses, the start position rst' of a clause detected by VAD may be earlier or later than the annotated start position rst. Similarly, the end position ret' of a clause detected by VAD may also appear before or after the annotated end position. Thus, based on the degree of difference between the detected position and the annotated position of the same clause, the accuracy of the VAD method of the speech recognition model can be quantified.

[0090] For example, for each clause, determine the length of the clause detected by VAD and the length of the annotated clause, calculate the difference between these two lengths, and finally calculate the average value of the length differences of each clause in the test speech as a measure of the accuracy of the VAD method in the speech recognition model.

[0091] During the speech processing process, the same clause may be divided into multiple sub-clauses by VAD. In order to align the sent clause and the received clause, so as to obtain the delay time by using the sending time parameters and receiving time parameters of the same clause, in the embodiments of the present disclosure, the following method can be used to determine multiple sub-clauses of the annotated clause in the speech recognition result. As Figure 8As shown, for each received clause, there is a corresponding start reception time and end reception time. For each clause Q sent, obtain the end reception time ret-1' of the first received clause (i.e., the detection clause) of this clause. If this end reception time ret-1' is before the end transmission time ret of clause Q, it indicates that clause Q has been divided into multiple clauses by VAD. The received clauses after the first received clause (referred to as the second received clause for short) can be obtained. Similarly, compare the end reception time ret-2' of the second received clause with the end transmission time ret of clause Q. If the end reception time ret-2' of the second received clause is after the end reception time of clause Q, it indicates that clause Q is divided into two clauses, namely the first received clause and the second received clause.

[0092] If the end reception time ret-2' of the second received clause is still before the end transmission time ret of clause Q, it indicates that clause Q is divided into at least 3 clauses, then continue to detect the next detection clause until the end reception time ret-n' of the detection clause n is after the end transmission time ret of clause Q. Thus, multiple clauses of clause Q are obtained.

[0093] In order to accurately determine the response time and hard delay, in the embodiments of the present disclosure, the reception time parameter of the speech recognition result of each clause in the test speech can be implemented as Figure 9 shown, including the following steps:

[0094] S901, for each clause, in the case where the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is divided into multiple clauses during the speech processing, obtain the start reception time of the speech recognition result of each clause in the multiple clauses.

[0095] S902, from the start reception times of the speech recognition results of each clause, screen out the start reception time that is after the start transmission time of the clause and is the closest to the start transmission time of the clause.

[0096] S903, determine the start reception time of the speech processing result of the clause as the screened start reception time.

[0097] For example, if clause Q is divided into 2 detection clauses and the start reception time of the first detection clause is earlier than the start transmission time of clause Q, then the start reception time of the second detection clause is used as the start reception time of the speech recognition result of clause Q.

[0098] Thus, in the embodiments of the present disclosure, when the same clause is divided into multiple sub-clauses during the voice processing, the start reception time of the voice processing result of the clause can be determined based on the start reception times of these multiple sub-clauses, so as to obtain the accurate start reception time of the speech recognition result and improve the accuracy of the test.

[0099] Correspondingly, when a clause is divided into multiple sub-clauses, the end reception time of the speech recognition result of the clause is obtained based on the following method, as Figure 9 shown, including:

[0100] S904. For each clause, when the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is divided into multiple sub-clauses during the voice processing, obtain the end reception time of the speech recognition result of the last sub-clause among the multiple sub-clauses.

[0101] S905. Determine the end reception time of the speech recognition result of the clause as the end reception time of the voice processing result of the clause.

[0102] Thus, in the embodiments of the present disclosure, when the same clause is divided into multiple sub-clauses during the voice processing, the introduction reception time of the voice processing result of the clause can be determined based on the end reception times of these multiple sub-clauses, so as to obtain the accurate end reception time and improve the accuracy of the test.

[0103] In some embodiments, in the test metric of the delay time, in order to accurately evaluate the response time and the hard delay, the embodiments of the present disclosure will exclude some clauses with errors. For example, for each clause, when the clause meets the specified conditions, it is determined that the clause does not participate in the calculation of the delay time;

[0104] The specified conditions include: the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause; or,

[0105] The specified conditions include: the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is not divided into multiple sub-clauses during the voice processing.

[0106] Thus, in the embodiments of the present disclosure, the clauses with the start reception time earlier than the start transmission time can be excluded, so that the delay time is not determined by these clauses, which can improve the accuracy of the delay time and the accuracy of the test result.

[0107] In another embodiment, for each clause, when the end reception time of the speech recognition result of the clause is between the start transmission time and the end transmission time of the (n)th clause after the clause, it is determined that the n clauses after the clause do not participate in the calculation of the delay time, where n is a positive integer.

[0108] As Figure 10 shown, assume that the start reception time of clause Q1 is before that of clause Q2, but the end reception time reQ1' of Q1 falls between the start transmission time and the end transmission time of clause Q2. Then clause Q2 is discarded, that is, clause Q2 does not participate in the calculation of the delay time. Assume that the end reception time reQ1' of clause Q1 falls between the start transmission time and the end transmission time of Q3. Then both clause Q2 and clause Q3 are discarded and neither participates in the calculation of the delay time.

[0109] Thus, in the embodiments of the present disclosure, clauses that are not suitable for determining the delay time can be discovered. When determining the delay time, the transmission time parameters and end time parameters of these clauses are not used, thereby improving the accuracy of the delay time calculation and the accuracy of the test results.

[0110] For ease of understanding, the following takes a product in which the client is responsible for collecting voice and sending it to the cloud for voice processing as an example to illustrate the method for testing the long voice processing ability of the cloud in this product.

[0111] In this embodiment, multiple long voices can be collected based on the client as test voices. Then, for each test voice, the annotation positions of each clause therein are marked to obtain a test set composed of multiple test voices.

[0112] After that, the test set is initialized, that is, the byte positions of the annotation positions of each clause in the test voice are determined using formula (1). Then as Figure 11 shown, the simulation end screens out the test voices from the test set and sends them to the cloud. While sending the test voices packet by packet, the simulation end records the start transmission time of each clause according to the byte position of the start position of each clause, and records the end transmission time of each clause based on the byte position of the end position of each clause.

[0113] In the cloud, a speech recognition model is deployed. While receiving the remaining data of the test voice, the speech recognition model segments each received test voice into clauses through VAD to obtain detected clauses, and then performs speech recognition on the detected clauses, and sends the recognition results packet by packet to the simulation end.

[0114] The simulation end, based on the start and end identifiers of each detected clause in the received packet, identifies the first received packet of each detected clause, marks the time when the first received packet is received to obtain the start reception time of the detected clause. Similarly, when the last received packet of the detected clause is received, the time is marked to obtain the end reception time of the detected clause.

[0115] The labeled positions of the test voice represent different clauses (hereinafter referred to as labeled clauses). When the first detected clause is received, the start reception time rs1' of the first detected clause is compared with the start transmission time s1 of the first labeled clause. If the start reception time rs1' of the first detected clause is before the start transmission time s1 of the first labeled clause, then the first detected clause will be discarded.

[0116] In another case, as Figure 12 shown, if rs1' is after s1 and the end reception time re1' of the first detected clause is before the end transmission time e1 of the first labeled clause, then the end reception time re2' of the second detected clause is checked. As Figure 12 shown, if the end reception time re2' of the second detected clause is after the end reception time e1 of the first labeled clause and before the start transmission time s2 of the second labeled clause, then the start reception time rs1' of the first detected clause and the end reception time re2' of the second detected clause are used to calculate the response time and hard delay of the first labeled clause.

[0117] As Figure 13 shown, S1 and e1 represent the start transmission time and end transmission time of the first labeled clause, and S2 and e2 represent the start transmission time and end transmission time of the second labeled clause. If the end reception time re2' of the second detected clause is between the start transmission time and end transmission time of the second labeled clause, then the second labeled clause is discarded and does not participate in the calculation of the response time and hard delay. And so on, the reception time parameters of the speech recognition results of each labeled clause participating in the calculation are obtained, and the response time and hard delay are calculated therefrom.

[0118] In addition, in addition to the delay time, the word accuracy and sentence accuracy of the speech recognition results can be calculated in the embodiments of the present disclosure to measure the performance of speech processing. The word accuracy can be understood as the ratio between the number of correctly recognized characters and the total number of characters in the test voice, and the sentence accuracy can be understood as the ratio of the number of correctly recognized clauses to all clauses in the test voice. For a sentence, if there is one character recognition error, then the sentence is not correctly recognized.

[0119] In other embodiments, the simulation end can also send multiple test voices simultaneously using multiple threads, thereby testing the concurrency ability of the cloud.

[0120] Based on the same technical concept, the embodiments of the present disclosure also provide a speech test device, as Figure 14 shown, which is a schematic structural diagram of the device, including:

[0121] A sending module 1401, configured to send multiple clauses in the test voice sentence by sentence and record the transmission time parameters of each clause;

[0122] A receiving module 1402, configured to receive the speech recognition results of the test speech sentence by sentence, and record the reception time parameter of the speech processing result of each clause;

[0123] An evaluation module 1403, configured to determine a delay time based on the transmission time parameter of each clause and the reception time parameter of the speech processing result of each clause.

[0124] In some embodiments, on the basis of Figure 14 as Figure 15 shown, the transmission time parameter includes a start transmission time, the reception time parameter includes a start reception time, the delay time includes a response time, and the evaluation module 1403 includes a response time determination unit 1501, configured to:

[0125] For each clause, determine a first time difference between the start transmission time of the clause and the start reception time of the speech recognition result of the clause;

[0126] Perform statistical analysis on the first time differences corresponding to each clause respectively to obtain the response time.

[0127] In some embodiments, the transmission time parameter includes an end transmission time, the reception time parameter includes an end reception time, the delay time includes a hard delay, and the evaluation module 1403 includes a hard delay determination unit 1502, configured to:

[0128] For each clause, determine a second time difference between the end transmission time of the clause and the end reception time of the speech recognition result of the clause;

[0129] Perform statistical analysis on the second time differences corresponding to each clause respectively to obtain the hard delay.

[0130] In some embodiments, as Figure 15 shown, the device further includes a first rejection module 1503, configured to:

[0131] For each clause, when the clause meets a specified condition, determine that the clause does not participate in the calculation of the delay time;

[0132] The specified condition includes: the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause; or,

[0133] The specified condition includes: the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is not divided into multiple sub-clauses during the speech processing process.

[0134] In some embodiments, the receiving module 1402 is configured to:

[0135] For each clause, when the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause, and the clause is divided into multiple sub-clauses during speech processing, obtain the start reception time of the speech recognition result of each sub-clause among the multiple sub-clauses;

[0136] From the start reception times of the speech recognition results of the respective sub-clauses, filter out the start reception time that is after the start transmission time of the clause and is the closest to the start transmission time of the clause;

[0137] Determine the start reception time of the speech processing result of the clause as the filtered start reception time.

[0138] In some embodiments, the receiving module 1402 is configured to:

[0139] For each clause, when the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause, and the clause is divided into multiple sub-clauses during speech processing, obtain the end reception time of the speech recognition result of the last sub-clause among the multiple sub-clauses;

[0140] Determine the end reception time of the speech processing result of the clause as the end reception time of the speech recognition result of the last sub-clause.

[0141] In some embodiments, as Figure 15 shown, the apparatus further includes a second elimination module 1504, configured to:

[0142] For each clause, when the end reception time of the speech recognition result of the clause is between the start transmission time and the end transmission time of the (n)th clause after the clause, determine that the (n) clauses after the clause do not participate in the calculation of the delay time, where n is a positive integer.

[0143] In some embodiments, the sending module 1401 is configured to:

[0144] Obtain the marked position of each clause in the test speech;

[0145] Based on the marked position of each clause, record the transmission time parameter of each clause.

[0146] In some embodiments, the sending module 1401 is configured to:

[0147] Based on the starting position of each clause, record the start transmission time in the transmission time parameter of each clause;

[0148] Based on the end position of each clause, record the end transmission time in the transmission time parameter of each clause.

[0149] In the embodiments of the present disclosure, for the speech processing of long speech, the transmission time parameter and the reception time parameter of each clause can be recorded, thereby implementing the delay time required for speech processing of long speech, filling the gap in the absence of a delay time test method in the case of long speech.

[0150] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0151] Figure 16 FIG. shows a schematic block diagram of an exemplary electronic device 1600 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0152] As Figure 16 shown, the electronic device 1600 includes a computing unit 1601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1602 or a computer program loaded from a storage unit 1608 into a random access memory (RAM) 1603. In the RAM 1603, various programs and data required for the operation of the electronic device 1600 can also be stored. The computing unit 1601, the ROM 1602, and the RAM 1603 are connected to each other through a bus 1604. An input / output (I / O) interface 1605 is also connected to the bus 1604.

[0153] A plurality of components in the electronic device 1600 are connected to the I / O interface 1605, including: an input unit 1606, such as a keyboard, a mouse, etc.; an output unit 1607, such as various types of displays, speakers, etc.; a storage unit 1608, such as a magnetic disk, an optical disk, etc.; and a communication unit 1609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1609 allows the electronic device 1600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0154] The computing unit 1601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1601 executes the voice test method described above. In some embodiments, the voice test method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1600 via the ROM 1602 and / or the communication unit 1609. When the computer program is loaded into the RAM 1603 and executed by the computing unit 1601, one or more steps of the voice test method can be executed. Alternatively, in other embodiments, the computing unit 1601 can be configured to execute the voice test method in any other suitable way (e.g., by means of firmware).

[0155] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0157] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0158] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0159] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0160] A computer system can include a client and a server. The client and the server are generally far apart from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server incorporating a blockchain.

[0161] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0162] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A voice test method, comprising: Sending multiple clauses in the test voice sentence by sentence, and recording the sending time parameters of each clause; The sending time parameters include: start sending time and end sending time; Receiving the voice recognition results of the test voice sentence by sentence, and recording the receiving time parameters of the voice processing results of each clause; the receiving time parameters include: start receiving time and end receiving time; Determining a delay time based on the sending time parameters of each clause and the receiving time parameters of the voice processing results of each clause; the delay time includes: response time and hard delay; The response time is determined based on the start receiving time and the start sending time; The hard delay is determined based on the end receiving time and the end sending time.

2. The method according to claim 1, wherein the determining the delay time based on the sending time parameters of each clause and the receiving time parameters of the voice processing results of each clause comprises: For each clause, determining a first time difference between the start sending time of the clause and the start receiving time of the voice recognition result of the clause; Performing statistical analysis on the first time differences corresponding to each clause to obtain the response time.

3. The method according to claim 1, wherein the determining the delay time based on the sending time parameters of each clause and the receiving time parameters of the voice processing results of each clause comprises: For each clause, determining a second time difference between the end sending time of the clause and the end receiving time of the voice recognition result of the clause; Performing statistical analysis on the second time differences corresponding to each clause to obtain the hard delay.

4. The method according to claim 2, further comprising: For each clause, when the clause meets a specified condition, determining that the clause does not participate in the calculation of the delay time; The specified condition includes: the start receiving time of the voice recognition result of the clause is earlier than the start sending time of the clause; or, The specified condition includes: the start receiving time of the voice recognition result of the clause is earlier than the start sending time of the clause and the clause is not divided into multiple sub - clauses during the voice processing.

5. The method according to claim 2, wherein the recording the receiving time parameters of the voice processing results of each clause comprises: For each clause, when the start receiving time of the voice recognition result of the clause is earlier than the start sending time of the clause and the clause is divided into multiple sub - clauses during the voice processing, obtaining the start receiving times of the voice recognition results of each sub - clause among the multiple sub - clauses; From the start receiving times of the voice recognition results of each sub - clause, screening out the start receiving time that is after the start sending time of the clause and is the closest to the start sending time of the clause; Determining the screened start receiving time as the start receiving time of the voice processing result of the clause.

6. The method according to claim 3, wherein the recording the receiving time parameters of the voice processing results of each clause comprises: For each clause, when the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is divided into multiple sub-clauses during speech processing, obtain the end reception time of the speech recognition result of the last sub-clause among the multiple sub-clauses; Determine the end reception time of the speech recognition result of the last sub-clause as the end reception time of the speech processing result of the clause.

7. The method according to claim 3 or 6, further comprising: For each clause, when the end reception time of the speech recognition result of the clause is between the start transmission time and the end transmission time of the (n + 1)-th clause after the clause, determine that the n clauses after the clause do not participate in the calculation of the delay time, where n is a positive integer.

8. The method according to claim 1, wherein the recording of the transmission time parameter of each clause comprises: Obtain the annotation position of each clause in the test speech; Based on the annotation position of each clause, record the transmission time parameter of each clause.

9. The method according to claim 8, wherein the annotation position comprises a start position and an end position, and the recording of the transmission time parameter of each clause based on the annotation position of each clause comprises: Based on the start position of each clause, record the start transmission time in the transmission time parameter of each clause; Based on the end position of each clause, record the end transmission time in the transmission time parameter of each clause.

10. A speech test device, comprising: A transmission module, configured to sequentially transmit multiple clauses in a test speech and record the transmission time parameter of each clause; The transmission time parameter comprises: a start transmission time and an end transmission time; A reception module, configured to sequentially receive the speech recognition results of the test speech and record the reception time parameter of the speech processing result of each clause; the reception time parameter comprises: a start reception time and an end reception time; An evaluation module, configured to determine a delay time based on the transmission time parameter of each clause and the reception time parameter of the speech processing result of each clause; the delay time comprises: a response time and a hard delay; The response time is determined based on the start reception time and the start transmission time; The hard delay is determined based on the end reception time and the end transmission time.

11. The device according to claim 10, wherein the evaluation module comprises a response time determination unit, configured to: For each clause, determine a first time difference between the start transmission time of the clause and the start reception time of the speech recognition result of the clause; Perform statistical analysis on the first time differences corresponding to each clause to obtain the response time.

12. The device according to claim 10, wherein the evaluation module comprises a hard delay determination unit, configured to: For each clause, determine a second time difference between the end transmission time of the clause and the end reception time of the speech recognition result of the clause; Perform statistical analysis on the second time differences corresponding to each clause to obtain the hard delay.

13. The apparatus according to claim 11 further includes a first rejection module, configured to: For each clause, when the clause meets the specified condition, determine that the clause does not participate in the calculation of the delay time; The specified conditions include: The start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause; Or, The specified condition includes: the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause, and the clause is not divided into multiple sub-clauses during the speech processing.

14. The receiving module of the apparatus according to claim 11 is configured to: For each clause, when the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is divided into multiple sub-clauses during the speech processing, obtain the start reception time of the speech recognition result of each sub-clause among the multiple sub-clauses; From the start reception times of the speech recognition results of the respective sub-clauses, filter out the start reception time that is after the start transmission time of the clause and is the closest to the start transmission time of the clause; Determine the start reception time of the speech processing result of the clause as the filtered start reception time.

15. The receiving module of the apparatus according to claim 12 is configured to: For each clause, when the start reception time of the speech recognition result of the clause is earlier than the start transmission time of the clause and the clause is divided into multiple sub-clauses during the speech processing, obtain the end reception time of the speech recognition result of the last sub-clause among the multiple sub-clauses; Determine the end reception time of the speech recognition result of the last sub-clause as the end reception time of the speech processing result of the clause.

16. The apparatus according to claim 12 or 15 further includes a second rejection module, configured to: For each clause, when the end reception time of the speech recognition result of the clause is between the start transmission time and the end transmission time of the nth clause after the clause, it is determined that the n clauses after the clause do not participate in the calculation of the delay time, where n is a positive integer.

17. The sending module of the apparatus according to claim 10 is configured to: Obtain the marked position of each clause in the test speech; Based on the marked position of each clause, record the transmission time parameter of each clause.

18. The sending module of the apparatus according to claim 17 is configured to: Based on the starting position of each clause, record the start transmission time in the transmission time parameter of each clause; Based on the ending position of each clause, record the end transmission time in the transmission time parameter of each clause.

19. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-9.

21. A computer program product, comprising a computer program, where the computer program implements the method according to any one of claims 1-9 when executed by a processor.

Images