Voice data transmission method, Internet of Things device and storage medium
By timing in the voice recognition device and determining the upload node of the voice recognition command in advance based on the cache queue status, the problem of extended device response time is solved, and the efficiency and user experience of voice recognition are improved.
Patent Information
- Application Number
- CN202011169545.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-27
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-10-27
AI Technical Summary
After the user stops speaking, existing voice recognition devices need to wait for the mute detection time and the remaining time of the previous transmission process, resulting in an extended voice response time and affecting the user experience.
By timing when the user's voice is detected to be muted, when the preprocessing time is reached, the upload node of the voice recognition command is determined according to the transmission status of the cache queue, and when the voice data and instructions in the cache queue are reached, the voice data and instructions in the cache queue are combined with the voice data and the commands when the node is reached. Upload to server.
It reduces the delay in voice data transmission, improves the server's response speed to voice recognition, and improves the user's voice control experience.
Smart Images

Figure CN112331198B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of speech recognition, and in particular to a speech data transmission method, an Internet of Things device and a storage medium. Background Art
[0002] At present, with the continuous development of speech recognition technology, IoT devices are becoming increasingly intelligent. When an IoT device receives a user's voice command, it needs to determine whether the user has stopped speaking. When it is determined that the user has stopped speaking, it sends a speech recognition command to the server to recognize the user's voice, and returns the recognition result to the IoT device to execute the corresponding command.
[0003] The existing voice end detection method mainly waits for a period of time when the voice signal is detected as silence for the first time, that is, when the user stops speaking, and continues to detect silence during this period of time to determine that the user has stopped speaking. When it is determined that the user has stopped speaking, all the voice data cached in the device can be uploaded to the server and the server can be requested to perform voice recognition. However, when it is determined that the user has stopped speaking, the cache queue may still be uploading the voice data cached last time, so the device needs to wait until the previous voice transmission process is over before uploading the last voice data. That is, after the user stops speaking, the IoT device sends the voice recognition command to the server. The response time is the sum of the waiting time for silence detection and the remaining time of the previous transmission process, which greatly increases the response time of the user's voice, seriously affecting the voice response speed of the device and reducing the user's voice control experience. Summary of the invention
[0004] The main purpose of the present invention is to provide a voice data transmission method, an Internet of Things device and a storage medium, aiming to solve the problem of slow existing voice response speed.
[0005] To achieve the above object, the present invention provides a voice data transmission method, which is applied to an Internet of Things device, and the method comprises the following steps:
[0006] When the user's voice data is detected to be silent, the duration of the silence is measured;
[0007] When the continuous silence duration reaches the preprocessing duration, determining the uploading node of the voice recognition instruction according to the transmission state of the cache queue, wherein the cache queue is used to cache the voice data;
[0008] When the uploading node of the speech recognition instruction is reached, the speech data and the speech recognition instruction in the cache queue are uploaded to the server.
[0009] Optionally, when the continuous silence duration reaches the preprocessing duration, the step of determining the uploading node of the speech recognition instruction according to the transmission state of the cache queue includes:
[0010] When the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue;
[0011] When the cache queue is in a waiting-for-transmission state, the cache queue is controlled to suspend the transmission of voice data, the detected new voice data is stored in the cache queue, and the upload node of the voice recognition instruction is determined to be a continuous silence duration that reaches a voice stop duration, wherein the voice stop duration is greater than the preprocessing duration.
[0012] Optionally, after the step of controlling the cache queue to suspend transmission of voice data and storing the detected new voice data in the cache queue, the method further includes:
[0013] Determining whether the voice data in the cache queue reaches a preset cache limit;
[0014] When the voice data in the cache queue reaches a preset cache limit, the voice data in the cache queue is uploaded to the server.
[0015] Optionally, when the continuous silence duration reaches the preprocessing duration, the step of determining the uploading node of the speech recognition instruction according to the transmission state of the cache queue includes:
[0016] When the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue;
[0017] When the cache queue is in the transmission state, the detected new voice data is stored in the cache queue, and the uploading node of the voice recognition instruction is determined to be in the transmission state and ends.
[0018] Optionally, when the cache queue is in the transmission state, the step of determining that the uploading node of the speech recognition instruction is in the transmission state ends includes:
[0019] When the cache queue is in the transmission state, determining that the uploading node of the speech recognition instruction is in the end of the transmission state and the continuous silence duration reaches the speech stop duration;
[0020] Accordingly, when the voice recognition instruction upload node is reached, the step of uploading the voice data and the voice recognition instruction in the cache queue to the server includes:
[0021] When the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, all voice data and voice recognition instructions in the buffer queue are uploaded to the server.
[0022] Optionally, when the continuous silence duration reaches the voice stop duration and the cache queue transmission state ends, the step of uploading all voice data and voice recognition instructions in the cache queue to the server includes:
[0023] When the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, determining whether the length of the voice data in the buffer queue is greater than the voice stop duration;
[0024] If the length of the voice data in the cache queue is greater than the voice stop duration, the voice data in the cache queue corresponding to the continuous silence duration is discarded, and the remaining voice data and voice recognition instructions in the cache queue are uploaded to the server;
[0025] If the length of the voice data in the cache queue is less than or equal to the voice stop duration, all the voice data in the cache queue are discarded, and the voice recognition instruction is uploaded to the server.
[0026] Optionally, when the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, the step of determining whether the length of the voice data in the buffer queue is greater than the voice stop duration includes:
[0027] When the continuous silence duration reaches the voice stop duration, determining whether the cache queue transmission state is ended;
[0028] If the transmission state of the cache queue has ended, determining whether the length of the voice data in the cache queue is greater than the voice stop duration;
[0029] If the transmission state of the buffer queue is not ended, the buffering of voice data to the buffer queue is stopped, and it is determined whether the length of the voice data in the buffer queue is greater than the voice stop duration.
[0030] Optionally, after the step of timing the duration of continuous silence when detecting that the voice data of the user is silent, the method further includes:
[0031] When it is detected that the user voice is not silent and the continuous silence duration does not reach the voice stop duration, the continuous silence duration is reset to zero and the timing is stopped.
[0032] In addition, to achieve the above-mentioned purpose, the present invention also provides an Internet of Things device, which includes a memory, a processor, and a voice data transmission program stored in the memory and run on the processor, wherein: when the voice data transmission program is executed by the processor, the steps of the voice data transmission method as described above are implemented.
[0033] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a voice data transmission program is stored, and when the voice data transmission program is executed by a processor, the steps of the voice data transmission method described above are implemented.
[0034] A voice data transmission method, an Internet of Things device, and a storage medium proposed in an embodiment of the present invention can detect the user's voice data, start timing when it is detected that the user's voice is silent, and determine the upload node of the voice recognition instruction in advance according to the transmission status of the cache queue when the timing duration reaches the preprocessing duration, and send the voice data and the voice recognition instruction in the cache queue to the server together when they reach the upload node. Determining the upload node of the voice recognition instruction in advance at the node when the continuous silence duration reaches the preprocessing duration can avoid the transmission instruction waiting time being too long when it is determined that the user has stopped speaking, reduce the transmission delay of the voice data, and improve the response speed of the server for voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;
[0036] Figure 2 A schematic diagram of a flow chart of a first embodiment of a voice data transmission method according to the present invention;
[0037] Figure 3 A schematic diagram of a flow chart of a second embodiment of a voice data transmission method according to the present invention;
[0038] Figure 4 A schematic diagram of a flow chart of a third embodiment of a voice data transmission method according to the present invention;
[0039] Figure 5 A flowchart of a fourth embodiment of a voice data transmission method according to the present invention;
[0040] Figure 6 A flowchart of a fifth embodiment of a voice data transmission method according to the present invention;
[0041] Figure 7 A flowchart of a sixth embodiment of a voice data transmission method according to the present invention;
[0042] Figure 8 A flowchart of a seventh embodiment of a voice data transmission method according to the present invention;
[0043] Fig. 9 This is a flow chart of an eighth embodiment of the voice data transmission method of the present invention;
[0044] Fig.10A schematic diagram of time nodes in an embodiment of a voice data transmission method of the present invention;
[0045] Fig.11 The figure is a schematic diagram of silence detection of the voice data transmission method of the present invention.
[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0048] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.
[0049] The terminal in the embodiment of the present invention may be an Internet of Things device, and the Internet of Things device may be an intelligent device such as an air conditioner, a smart speaker, a refrigerator, a smart lamp, etc.
[0050] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0051] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the hardware device is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the hardware device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the hardware device can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.
[0052] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0053] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a voice data transmission program.
[0054] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and perform data communication with the client; and the processor 1001 can be used to call the voice data transmission program stored in the memory 1005 and perform the following operations:
[0055] When the user's voice data is detected to be silent, the duration of the silence is measured;
[0056] When the continuous silence duration reaches the preprocessing duration, determining the uploading node of the voice recognition instruction according to the transmission state of the cache queue, wherein the cache queue is used to cache the voice data;
[0057] When the uploading node of the speech recognition instruction is reached, the speech data and the speech recognition instruction in the cache queue are uploaded to the server.
[0058] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0059] When the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue;
[0060] When the cache queue is in a waiting-for-transmission state, the cache queue is controlled to suspend the transmission of voice data, the detected new voice data is stored in the cache queue, and the upload node of the voice recognition instruction is determined to be a continuous silence duration that reaches a voice stop duration, wherein the voice stop duration is greater than the preprocessing duration.
[0061] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0062] Determining whether the voice data in the cache queue reaches a preset cache limit;
[0063] When the voice data in the cache queue reaches a preset cache limit, the voice data in the cache queue is uploaded to the server.
[0064] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0065] When the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue;
[0066] When the cache queue is in the transmission state, the detected new voice data is stored in the cache queue, and the uploading node of the voice recognition instruction is determined to be in the transmission state and ends.
[0067] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0068] When the cache queue is in the transmission state, determining that the uploading node of the speech recognition instruction is in the end of the transmission state and the continuous silence duration reaches the speech stop duration;
[0069] When the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, all voice data and voice recognition instructions in the buffer queue are uploaded to the server.
[0070] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0071] When the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, determining whether the length of the voice data in the buffer queue is greater than the voice stop duration;
[0072] If the length of the voice data in the cache queue is greater than the voice stop duration, the voice data in the cache queue corresponding to the continuous silence duration is discarded, and the remaining voice data and voice recognition instructions in the cache queue are uploaded to the server;
[0073] If the length of the voice data in the cache queue is less than or equal to the voice stop duration, all the voice data in the cache queue are discarded, and the voice recognition instruction is uploaded to the server.
[0074] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0075] When the continuous silence duration reaches the voice stop duration, determining whether the cache queue transmission state is ended;
[0076] If the transmission state of the cache queue has ended, determining whether the length of the voice data in the cache queue is greater than the voice stop duration;
[0077] If the transmission state of the buffer queue is not ended, the buffering of voice data to the buffer queue is stopped, and it is determined whether the length of the voice data in the buffer queue is greater than the voice stop duration.
[0078] Further, the processor 1001 may call the voice data transmission program stored in the memory 1005, and further perform the following operations:
[0079] When it is detected that the user voice is not silent and the continuous silence duration does not reach the voice stop duration, the continuous silence duration is reset to zero and the timing is stopped.
[0080] The specific embodiments of the present invention applying the Internet of Things device are basically the same as the embodiments of the following application voice data transmission method, and will not be described in detail here.
[0081] Please refer to Figure 2 , Figure 2 The figure is a flow chart of the first embodiment of the voice data transmission method of the present invention, wherein the voice data transmission method is applied to an Internet of Things device, and the voice data transmission method comprises the following steps:
[0082] Step S10, when it is detected that the user's voice data is silent, timing the duration of continuous silence;
[0083] The terminal of this embodiment can be an IoT device, which has at least a microphone and a communication module, and can collect the voice of the user and upload the user's voice data to the server for voice recognition. After the server recognizes the voice data and returns the recognition result, the IoT device can execute the corresponding user instruction according to the recognition result.
[0084] When the user is speaking, the IoT device will continue to collect the voice of the user until the IoT device determines that the user has stopped speaking, and then send the last voice data to the server, and send a voice recognition instruction to the server at the same time as sending the voice data for the last time, so that the server can perform voice recognition. The way to determine that the user has stopped speaking is to count the continuous silence time when the user's voice data is detected to be silent, and determine that the user has stopped speaking only after the continuous silence time of the user's voice reaches the voice stop time. Since the device needs to consume a certain transmission time each time it sends voice data to the server, the transmission time is usually about 500ms, and the transmission time is mainly determined by the hardware link and WIFI speed, and has little to do with the amount of data transmitted each time. That is, if the device just enters the transmission state when it is determined that the user has stopped speaking and is ready to send the last voice data and voice recognition instruction, the voice recognition instruction will need to wait for about 500ms of transmission time before it can be transmitted, thereby delaying the time node of the server receiving the voice recognition instruction, causing a delay in voice recognition.
[0085] Step S20, when the continuous silence duration reaches the preprocessing duration, determining the uploading node of the voice recognition instruction according to the transmission state of the cache queue, wherein the cache queue is used to cache voice data;
[0086] When the IoT device continuously monitors that the user's voice data is silent, the duration of continuous silence can be timed. When the continuous silence duration reaches the preprocessing duration, the upload node of the voice recognition instruction can be determined according to the current transmission status of the cache queue that caches the voice data, so that the voice recognition instruction and voice data can be quickly uploaded to the server when the continuous silence duration reaches the voice stop duration.
[0087] It is understandable that the preprocessing duration and the voice stop duration are preset, wherein the preprocessing duration is less than the voice stop duration. For example, the voice stop duration can be preset to 600ms and the preprocessing duration to 300ms. When the IoT device continuously monitors that the user's voice data is silent, the continuous silence duration is timed. When the continuous silence duration reaches 300ms, the upload node of the voice recognition instruction can be determined according to the transmission status of the current cache queue, and the voice recognition instruction can be uploaded according to the upload node of the voice recognition instruction after the continuous silence duration reaches 600ms, so that the server can quickly receive the recognition instruction after determining that the user has stopped speaking, thereby reducing the delay in voice recognition.
[0088] Step S30, when reaching the upload node of the voice recognition instruction, uploading the voice data and the voice recognition instruction in the cache queue to the server.
[0089] When the continuous silence duration reaches the preprocessing duration and the user's voice data is still silent, the continuous silence duration can be continuously counted until the upload node of the voice recognition instruction is reached, and the remaining voice data and the voice recognition instruction in the current cache queue can be uploaded to the server together. So that the server can perform voice recognition based on all the uploaded voice data. The data in the cache queue can be uploaded by uploading the voice data and the voice recognition instruction to the server through an http request.
[0090] It should be noted that, since confirming that the user has stopped speaking requires that the duration of continuous monitoring of silence reaches the speech stop duration, if the continuous silence duration has reached the speech stop duration when the upload node of the speech recognition instruction is reached, the voice data and speech recognition instructions in the cache queue can be immediately uploaded to the server; if the continuous silence duration has not reached the speech stop duration when the upload node of the speech recognition instruction is reached, the IoT device needs to wait until the continuous silence duration reaches the speech stop duration before uploading the voice data and speech recognition instructions in the cache queue to the server.
[0091] In this embodiment, by detecting the user's voice data, it is possible to start timing when it is detected that the user's voice is silent, and when the timing duration reaches the preprocessing duration, the upload node of the voice recognition instruction is determined in advance according to the transmission status of the cache queue, and when the upload node is reached, the voice data and the voice recognition instruction in the cache queue are sent to the server together. By determining the upload node of the voice recognition instruction in advance when the continuous silence duration reaches the preprocessing duration, it is possible to avoid waiting too long for the transmission instruction when it is determined that the user has stopped speaking, reduce the transmission delay of the voice data, and improve the response speed of the server for voice recognition.
[0092] Further, see Figure 3 , Figure 3 This is a flow chart of a second embodiment of a voice data transmission method according to the present invention. Figure 2 In the embodiment shown, in step S20, when the continuous silence duration reaches the preprocessing duration, the uploading node of the voice recognition instruction is determined according to the transmission state of the cache queue, and the cache queue is used to cache the voice data, including:
[0093] Step S21, when the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue;
[0094] Step S22, when the cache queue is in a waiting-for-transmission state, control the cache queue to suspend the transmission of voice data, store the detected new voice data in the cache queue, and determine that the upload node of the voice recognition instruction is a continuous silence duration that reaches a voice stop duration, wherein the voice stop duration is greater than the preprocessing duration.
[0095] In this embodiment, the cache queue can continuously receive the user voice data collected by the IoT device, and upload the voice data in the cache queue to the server during each transmission process. It is understandable that 108 packets of data can be stored in the cache queue, and the user voice data collected by the IoT is about 10 packets of data every 100ms, that is, the cache queue can store about 1000ms of voice data. The IoT device consumes about 500ms when transmitting each voice data. It is understandable that during the time interval consumed by transmitting voice data, the cache queue is in the transmission state, and during the time interval when the transmission of one voice data is completed and the next voice data transmission has not yet started, the cache queue is in the waiting state. When the cache queue is in the transmission state, the cache queue can upload the user voice data stored in it to the server, and can also receive new user voice data for storage, so that when the cache queue enters the next transmission state, the new user voice data received in the previous transmission state is uploaded; and when the cache queue is in the waiting state, the cache queue can receive new user voice data and upload the user voice data to the server when the state is converted to the transmission state. Among them, when the cache queue is in the transmission state, the IoT device is still collecting the user's voice data and storing it in the cache queue. The voice data stored in the cache queue is the new voice data. When the cache queue suspends uploading voice data to the server, the cache queue not only includes the voice data that should have been uploaded but suspended, but also includes the user's voice data that the IoT device continuously collects and stores. Compared with the voice data that is suspended, the user's voice data is the new voice data. The new voice data will also be stored in the cache queue until it reaches the upload node of the voice recognition instruction. The cache queue will upload the voice data that is suspended and the new voice data collected during the suspension period to the server.
[0096] When the IoT device times the duration of continuous silence to the preprocessing duration, the transmission status of the current cache queue can be obtained. If the current cache queue has just finished the last transmission process and enters the waiting state, the cache queue can be controlled to pause uploading voice data, and the newly detected voice data will continue to be stored in the cache queue. At this time, it can be determined that the upload node of the voice recognition instruction is when the continuous silence duration reaches the voice stop duration. The time node when the timing of the continuous silence duration reaches the preprocessing duration is the starting node, and the user voice data collected thereafter is the new voice data.
[0097] It is understandable that the IoT device determines the upload node when the timing of the continuous silence duration reaches the preprocessing duration, and the upload node is when the timing of the continuous silence duration reaches the voice stop duration, that is, the continuous silence duration should first reach the preprocessing duration and then reach the voice stop duration, and the voice stop duration is greater than the preprocessing duration.
[0098] It is understandable that when the cache queue is controlled to suspend uploading voice data, the voice data stored in the cache queue at this time is the user voice data detected during the last transmission process. On this basis, the cache queue can continue to store new voice data until the continuous silence duration reaches the voice stop duration. At this time, it can be determined that the user has stopped speaking, and all voice data and voice recognition instructions in the cache queue are uploaded to the server. For example, when the continuous silence duration reaches 300ms, if the cache queue is currently in a waiting state, and the previous transmission process takes about 500ms, then the cache queue currently stores about 500ms of voice data. At this time, new voice data can continue to be stored in the cache queue until the continuous silence duration reaches 600ms, and the voice data originally stored in the cache queue for about 500ms and the newly stored voice data within 300ms from the preprocessing duration to the voice stop duration can be uploaded to the server together with the voice recognition instruction. The above 300ms is the preprocessing duration, and 600ms is the voice stop duration. By setting the cache space of the cache queue, the cache queue can store at least one voice data cached during the voice transmission process and the voice data during the period from the preprocessing time to the voice stop time, so that when the continuous silence time reaches the voice stop time, the voice recognition command and the remaining voice data are immediately uploaded to the server, which fully reduces the transmission waiting time of the voice recognition command and reduces the delay of voice recognition.
[0099] Please refer to Fig.11 In one embodiment, S is the time point when the user speaks, A is the time point when the user stops speaking, and the voice data emitted by the user can be collected in the SA interval. M is the time point when the continuous silence duration reaches the preprocessing duration, and B is the time point when the continuous silence duration reaches the voice stop duration. When the user is detected to be silent at point A, it cannot be directly determined that the user has stopped speaking. It is necessary to start timing the silence duration from point A. At point M, it is necessary to determine the upload node of the voice recognition instruction. It is not until point B is reached that it is determined that the user has stopped speaking, and the voice recognition instruction is sent to the server according to the determined voice recognition instruction upload node.
[0100] exist Fig.10In the embodiment shown, the real-time state of the cache queue switches between the waiting state and the transmission state in sequence. There are two situations in this embodiment. In the first situation (1), the ba segment is in the transmission state, the ab' segment is in the waiting state, and the continuous silence duration reaches the preprocessing duration, that is, when it is in the M node, the cache queue is in the ab' segment, just ready to initiate a data upload, that is, at this time the cache queue is in the waiting state, and the voice data upload can be temporarily suspended at this time, that is, b' is moved back, and then the cache queue in the MB node interval will receive new voice data and cache it. When the continuous silence duration reaches the B node, it can be determined that the user silence is continuously detected in the AB interval, and the upload node of the voice recognition instruction is the C node. At this time, the C node coincides with the B node, that is, the C node is the C1 node. That is, if the cache queue is in the waiting state at the M node, the voice recognition instruction can be uploaded immediately when it reaches the B node.
[0101] In the second case, the ba segment and the b'a' segment are in the transmission state, the ab' segment and the a'b" segment are in the waiting state, and the continuous silence duration reaches the preprocessing duration, that is, when it is at the M node, the cache queue is in the b'a' segment. At this time, the cache queue is in the transmission state and is waiting for the transmission state to end. The time node when the transmission state ends can be C2. It can be understood that the C2 node can be before or after the B node. When the C2 node is before the B node, the data upload can be suspended when reaching the C2 node, and the voice recognition instruction can be uploaded to the server when reaching the B node. At this time, the upload node of the voice recognition instruction is the B node. When the C2 node is after the B node, even if it reaches the B node and determines that the user stops speaking, since the data upload process of the cache queue has not ended at this time, it is necessary to wait until the data upload ends, that is, when it reaches the C2 node, before the voice recognition instruction can be uploaded to the server. That is, if the cache queue is in the transmission state at the M node, the upload node of the voice recognition instruction is the latter node between the B node and the C2 node.
[0102] It should be noted that in the above embodiment, when the voice data uploaded by the cache queue at the same time as uploading the voice recognition instruction are all voice data after the A node, since the user is continuously detected to be silent after the A node, all voice data to be uploaded are silent data. At this time, all voice data can be discarded and the voice recognition instruction can be directly uploaded to the server. When the voice data uploaded in the cache queue at the same time as the voice recognition instruction includes the voice data before the A node, the voice data before the A node is uploaded to the server together with the voice recognition instruction, and the silent voice data received after the A node is discarded.
[0103] Further, see Figure 4 , Figure 4 This is a flow chart of a third embodiment of a method for transmitting voice data according to the present invention. Figure 3In the embodiment shown, the step S22, when the cache queue is in a waiting state, controls the cache queue to suspend the transmission of voice data, stores the detected new voice data in the cache queue, and determines that the upload node of the voice recognition instruction is the step of the continuous silence duration reaching the voice stop duration, further comprising:
[0104] Step S40, determining whether the voice data in the cache queue reaches a preset cache limit;
[0105] Step S41, when the voice data in the cache queue reaches a preset cache limit, the voice data in the cache queue is uploaded to the server.
[0106] In this embodiment, after controlling the cache queue to suspend the transmission of voice data and storing new voice data in the cache queue, it is also necessary to monitor whether the voice data stored in the cache queue reaches the preset cache limit. If the voice data in the cache queue reaches the preset cache limit, storing new voice data in the cache queue will cause the user's voice data to be lost, so the voice data in the current cache queue needs to be uploaded to the server immediately to avoid the loss of user voice data. It is understandable that at this time, the upload node of the voice recognition instruction needs to meet the continuous silence time to reach the voice stop time, and the transmission process corresponding to the voice data that currently reaches the preset cache limit ends.
[0107] It should be noted that, usually, the transmission process of voice data is about 500ms, but when a network jam or other failure occurs, the transmission process will be prolonged. At this time, the voice data stored in the cache queue during the suspension of voice data transmission is likely to reach the preset cache limit, and the voice data needs to be uploaded to the server in time to avoid voice data loss. The preset cache limit can be set to be related to the cache space of the cache queue, for example, it can be set to 90% of the cache space.
[0108] Further, see Figure 5 , Figure 5 This is a flow chart of a fourth embodiment of a method for transmitting voice data according to the present invention. Figures 2 to 4 In the embodiment shown, in step S20, when the continuous silence duration reaches the preprocessing duration, the uploading node of the voice recognition instruction is determined according to the transmission state of the cache queue, and the cache queue is used to cache the voice data, including:
[0109] Step S21, when the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue;
[0110] Step S23, when the cache queue is in the transmission state, the detected new voice data is stored in the cache queue, and the uploading node of the voice recognition instruction is determined to be in the transmission state and ends.
[0111] In this embodiment, when the IoT device counts the duration of continuous silence and reaches the preprocessing duration, the transmission status of the current cache queue can be obtained. If the current cache queue is in the transmission state, that is, when voice data is being uploaded to the server, the detected new voice data can continue to be stored in the cache queue, and at this time, it can be determined that the upload node of the voice recognition instruction is the end of the transmission state. That is, at the time point when the continuous silence duration reaches the preprocessing duration, the upload node of the voice recognition instruction is determined to be the end of this transmission process.
[0112] Further, see Figure 6 , Figure 6 This is a flow chart of a fifth embodiment of a method for transmitting voice data according to the present invention. Figure 5 In the embodiment shown, the step S23, when the cache queue is in the transmission state, storing the detected new voice data in the cache queue, and determining that the uploading node of the voice recognition instruction is in the transmission state ends, includes:
[0113] Step S50, when the cache queue is in the transmission state, storing the detected new voice data in the cache queue, determining that the uploading node of the voice recognition instruction is in the end of the transmission state and the continuous silence duration reaches the voice stop duration;
[0114] The step S30, when reaching the upload node of the speech recognition instruction, uploading the speech data and the speech recognition instruction in the cache queue to the server comprises:
[0115] Step S31, when the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, all voice data and voice recognition instructions in the buffer queue are uploaded to the server.
[0116] In this embodiment, when it is determined that the current cache queue is in the transmission state, that is, when voice data is being uploaded to the server, it can also be determined that the upload node of the voice recognition instruction is in the end of the transmission state and the continuous silence duration reaches the voice stop duration. That is, when the continuous silence duration reaches the preprocessing duration, the cache queue has two states, one is the waiting transmission state, that is, the previous transmission process is ended and the next transmission process is prepared; the other is the transmission state, that is, the transmission process is in progress. When the cache queue is in the waiting transmission state, it can be determined that the voice recognition instruction and the upload node of the last voice data upload are in the end of the continuous silence duration and the voice stop duration. When the cache queue is in the transmission state, it can be determined that the voice recognition instruction and the upload node of the last voice data upload are in the end of the transmission state and the continuous silence duration reaches the voice stop duration.
[0117] At the end of the current transmission process, if the continuous silence duration has reached the voice stop duration, it can be determined that the user has stopped speaking, and after the cache queue transmission state ends, the voice recognition instructions and the newly stored voice data in the cache queue during the transmission process are uploaded to the server. If the continuous silence duration does not reach the voice stop duration, it is necessary to wait until the continuous silence duration reaches the voice stop duration, that is, to determine that the user has stopped speaking before uploading the remaining voice data and voice recognition instructions in the cache queue to the server.
[0118] It should be noted that when the transmission process of the cache queue ends when the continuous silence duration reaches the preprocessing duration, if the continuous silence duration does not reach the voice stop duration, although it is necessary to wait for the continuous silence duration to reach the voice stop duration, since the user's voice data is silent, the user's voice data detected during the period from the end of the transmission process to the time when the continuous silence duration does not reach the voice stop duration can be discarded without being stored in the cache queue. When the continuous silence duration does not reach the voice stop duration, only the voice data stored in the cache queue needs to be uploaded to the server.
[0119] Further, see Figure 7 , Figure 7 This is a flow chart of a sixth embodiment of a voice data transmission method according to the present invention. Figure 6 In the embodiment shown, the step S31, when the continuous silence duration reaches the voice stop duration and the cache queue transmission state ends, the step of uploading all voice data and voice recognition instructions in the cache queue to the server includes:
[0120] Step S311, when the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, determining whether the length of the voice data in the buffer queue is greater than the voice stop duration;
[0121] Step S312: If the length of the voice data in the cache queue is greater than the voice stop duration, the voice data in the cache queue corresponding to the continuous silence duration is discarded, and the remaining voice data and the voice recognition instruction in the cache queue are uploaded to the server;
[0122] Step S313: If the length of the voice data in the cache queue is less than or equal to the voice stop duration, all the voice data in the cache queue are discarded, and the voice recognition instruction is uploaded to the server.
[0123] In this embodiment, when the continuous silence duration reaches the voice stop duration and the cache queue transmission state has ended, the current time node can be determined as the upload node of the voice recognition instruction. By determining the length of the voice data in the cache queue, it can be determined whether all the voice data stored in the cache queue are silent data. It can be understood that the length of the voice data in the cache queue is the duration of the user voice data. For example, when the user voice data in the cache queue can be played for 500 milliseconds, the length of the voice data is 500 milliseconds. If the length of the voice data in the cache queue is greater than the voice stop duration, since the time length corresponding to the user's silent data is the semantic stop duration, the voice data in the cache queue must contain user voice data other than silent data, and whether the silent data is uploaded to the server or not does not affect the server's voice recognition. For example, when the voice stop duration is 600ms and the length of the voice data in the cache queue is 650ms, it means that the voice data in the cache queue contains 50ms of user voice and 600ms of silent data. If the voice data cached in the cache queue is silent data during the time interval from 0 to the voice stop time, then when it is determined that there is non-silent user voice data in the cache queue, the voice data in the cache queue corresponding to the time interval from 0 to the voice stop time can be discarded. At this time, the remaining voice data in the cache queue is the user voice collected and cached before the user's voice data is detected to be silent. When the upload node of the voice recognition instruction is reached, the remaining voice data and the voice recognition instruction can be uploaded to the server together.
[0124] Similarly, if the length of the voice data in the cache queue is less than or equal to the voice stop duration, it means that all the voice data stored in the cache queue is silent data. At this time, all the voice data in the cache queue can be discarded and only the voice recognition instruction can be uploaded to the server.
[0125] Further, see Figure 8 , Figure 8 This is a flow chart of the seventh embodiment of the voice data transmission method of the present invention. Figure 7 In the embodiment shown, the step S311, when the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, the step of determining whether the voice data length in the buffer queue is greater than the voice stop duration includes:
[0126] Step S3111, when the continuous silence duration reaches the voice stop duration, determining whether the buffer queue transmission state is ended;
[0127] If the transmission state of the cache queue has ended, executing step S3112, determining whether the length of the voice data in the cache queue is greater than the voice stop time;
[0128] If the transmission state of the cache queue is not ended, step S3113 is executed to stop caching voice data to the cache queue, and step S3112 is executed to determine whether the length of the voice data in the cache queue is greater than the voice stop duration.
[0129] In this embodiment, when the continuous silence duration reaches the speech stop duration and the buffer queue transmission state ends, it is the uploading node of the speech recognition instruction. There are two situations: the continuous silence duration reaches the speech stop duration first, and the buffer queue transmission state enters the end state first.
[0130] If the continuous silence duration reaches the speech stop duration first, because the cache queue transmission state is still in the transmission state and has not ended, the upload node of the speech recognition instruction is the end of the transmission state of the cache queue. If the continuous silence duration does not reach the speech stop duration when the cache queue transmission state has ended, the upload node of the speech recognition instruction is the continuous silence duration reaching the speech stop duration.
[0131] It can be understood that when the cache queue transmission state first enters the end state, the upload node of the voice recognition instruction is when the continuous silence duration reaches the voice stop duration. At this time, it can be determined whether the length of the voice data stored in the cache queue is greater than the voice stop duration, thereby determining whether there is non-silent user voice data stored in the cache queue, and uploading the non-silent user voice data together with the voice recognition instruction.
[0132] When the continuous silence duration reaches the voice stop duration first, since the user voice data collected during the continuous silence duration is all silent data, it can be determined that the user has stopped speaking when the continuous silence duration reaches the voice stop duration, and the voice data collected after the voice stop duration is reached can be directly discarded and no longer stored in the cache queue. At this time, the voice data stored in the cache queue is the user voice data from the starting node of the cache queue's current transmission process to the continuous silence duration reaching the voice stop duration. That is, when the continuous silence duration reaches the voice stop duration, regardless of whether the transmission state of the cache queue is ended at this time, the cache queue stops caching voice data, and divides the cached voice data into silent data and non-silent data, discards the silent data, and uploads the non-silent data and the voice recognition instruction to the server when the upload node of the voice recognition instruction is reached.
[0133] Further, see Fig. 9 , Fig. 9 This is a flow chart of an eighth embodiment of the voice data transmission method of the present invention. Figures 2 to 8 In the embodiment shown, after the step S10 of timing the duration of continuous silence when detecting that the user's voice data is silent, the step further includes:
[0134] Step S60: When it is detected that the user's voice is not silent and the continuous silence duration does not reach the voice stop duration, the continuous silence duration is reset to zero and the timing is stopped.
[0135] In this embodiment, the continuous silence duration is measured only when the user's voice data is detected to be silent. If the user's voice is detected to be not silent when the continuous silence duration does not reach the voice stop duration during the timing process, it can be determined that the user has not stopped speaking. At this time, the continuous silence duration can be stopped and reset, and the user's voice data can be continuously detected until it is determined that the user has stopped speaking and the voice recognition instruction is uploaded to the server.
[0136] In addition, the present invention also provides a computer-readable storage medium on which a voice data transmission program is stored. The computer-readable storage medium may be Figure 1 The memory 20 in the terminal may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a plurality of instructions for enabling an Internet of Things device having a processor to execute the voice data transmission method described in each embodiment of the present invention.
[0137] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0138] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0139] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0140] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0141] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A voice data transmission method, characterized in that: Applied to an IoT device, the method comprises the following steps: When the user's voice data is detected to be silent, the duration of the silence is measured; When the continuous silence duration reaches the preprocessing duration, determining the uploading node of the voice recognition instruction according to the transmission state of the cache queue, wherein the cache queue is used to cache the voice data; When the voice recognition instruction upload node is reached, the voice data and the voice recognition instruction in the cache queue are uploaded to the server; The step of determining the uploading node of the speech recognition instruction according to the transmission state of the cache queue when the continuous silence duration reaches the preprocessing duration includes: When the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue; When the cache queue is in a waiting-for-transmission state, control the cache queue to suspend transmission of voice data, store the detected new voice data in the cache queue, and determine that the uploading node of the voice recognition instruction is when the continuous silence duration reaches the voice stop duration, wherein the voice stop duration is greater than the preprocessing duration; Or the step of determining the uploading node of the speech recognition instruction according to the transmission state of the cache queue when the continuous silence duration reaches the preprocessing duration includes: When the continuous silence duration reaches the preprocessing duration, obtaining the transmission status of the cache queue; When the cache queue is in the transmission state, the detected new voice data is stored in the cache queue, and the uploading node of the voice recognition instruction is determined to be in the transmission state and ends.
2. The voice data transmission method according to claim 1, characterized in that: After the step of controlling the cache queue to suspend the transmission of voice data and storing the detected new voice data in the cache queue, the following steps are further included: Determining whether the voice data in the cache queue reaches a preset cache limit; When the voice data in the cache queue reaches a preset cache limit, the voice data in the cache queue is uploaded to the server.
3. The voice data transmission method according to claim 1, characterized in that: The step of determining that the uploading node of the speech recognition instruction is in the transmission state when the cache queue is in the transmission state includes: When the cache queue is in the transmission state, determining that the uploading node of the speech recognition instruction is in the end of the transmission state and the continuous silence duration reaches the speech stop duration; Accordingly, when the voice recognition instruction upload node is reached, the step of uploading the voice data and the voice recognition instruction in the cache queue to the server includes: When the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, all voice data and voice recognition instructions in the buffer queue are uploaded to the server.
4. The voice data transmission method according to claim 3, characterized in that: The step of uploading all voice data and voice recognition instructions in the cache queue to the server when the continuous silence duration reaches the voice stop duration and the cache queue transmission state ends includes: When the continuous silence duration reaches the voice stop duration and the buffer queue transmission state ends, determining whether the length of the voice data in the buffer queue is greater than the voice stop duration; If the length of the voice data in the cache queue is greater than the voice stop duration, the voice data in the cache queue corresponding to the continuous silence duration is discarded, and the remaining voice data and voice recognition instructions in the cache queue are uploaded to the server; If the length of the voice data in the cache queue is less than or equal to the voice stop duration, all the voice data in the cache queue are discarded, and the voice recognition instruction is uploaded to the server.
5. The voice data transmission method according to claim 4, characterized in that: The step of determining whether the length of voice data in the cache queue is greater than the voice stop duration when the continuous silence duration reaches the voice stop duration and the cache queue transmission state ends comprises: When the continuous silence duration reaches the voice stop duration, determining whether the cache queue transmission state is ended; If the transmission state of the cache queue has ended, determining whether the length of the voice data in the cache queue is greater than the voice stop duration; If the transmission state of the buffer queue is not ended, the buffering of voice data to the buffer queue is stopped, and it is determined whether the length of the voice data in the buffer queue is greater than the voice stop duration.
6. The voice data transmission method according to any one of claims 1 to 5, characterized in that: After the step of timing the duration of continuous silence when the user's voice data is detected to be silent, the method further includes: When it is detected that the user voice is not silent and the continuous silence duration does not reach the voice stop duration, the continuous silence duration is reset to zero and the timing is stopped.
7. An Internet of Things device, characterized in that: The Internet of Things device includes a memory, a processor, and a voice data transmission program stored in the memory and executable on the processor, wherein: when the voice data transmission program is executed by the processor, the steps of the voice data transmission method according to any one of claims 1 to 6 are implemented.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a voice data transmission program, and when the voice data transmission program is executed by a processor, the steps of the voice data transmission method according to any one of claims 1 to 6 are implemented.
Citation Information
Cited By
Voice data transmission method based on cloud platform
CN121530957A
