Speech recognition method, apparatus, system, device, medium and program product

Abstract

The present disclosure provides a speech recognition method, which can be applied to the fields of cloud computing technology and artificial intelligence technology. The speech recognition method is applied to a user terminal, and the method comprises: acquiring network state data at a current time; based on the network state data at the current time and a preset first agent, dividing a collected user speech audio stream into frames to obtain audio slices, wherein the first agent is formed based on reinforcement learning; sending the audio slices to a speech recognition server; and receiving speech recognition information from the speech recognition server. The present disclosure also provides a speech recognition device, system, equipment, storage medium and program product.

Description

Technical Field

[0001] This disclosure relates to the fields of cloud computing technology and artificial intelligence technology, and specifically to a speech recognition method, apparatus, system, device, medium and program product. Background Technology

[0002] Cloud-based speech recognition is a technical solution that places speech at a remote location to perform speech recognition processing. Typically, this solution places the speech recognition engine on a cloud server and places the speech acquisition / transmission logic in the user terminal client. The client acquires the user's speech, performs some preprocessing, and then transmits it to the server where the recognition engine is located. After the server recognizes the speech, it returns the recognition result to the user terminal.

[0003] In this context, the existing cloud-based speech recognition logic servers are located in a central data center. If a city is far from the data center, this results in significant link latency. This latency is exacerbated when speech data is transmitted to the data center and then the recognized results are returned. Furthermore, the existing solution cannot guarantee a data flow transmission strategy that adapts to the current network conditions in complex network links, making it difficult to ensure the timeliness of speech data transmission. Summary of the Invention

[0004] In view of the above problems, this disclosure provides speech recognition methods, apparatus, systems, devices, media, and program products that improve the efficiency and stability of remote speech recognition.

[0005] According to a first aspect of this disclosure, a speech recognition method is provided, the method being applied to a user terminal, the method comprising: acquiring network status data at a current moment; dividing a collected user speech audio stream into frames based on the network status data at the current moment and a preset first intelligent agent to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; sending the audio slices to a speech recognition server; and receiving speech recognition information from the speech recognition server.

[0006] According to an embodiment of this disclosure, the step of segmenting the collected user voice audio stream into frames based on the current network state data and a preset first intelligent agent to obtain audio slices includes: acquiring a preset plurality of slice actions; evaluating and scoring the plurality of slice actions based on the current network state data and the preset first intelligent agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segmenting the user voice audio stream into frames based on the first slice action.

[0007] According to an embodiment of this disclosure, after receiving speech recognition information from the speech recognition server, the method further includes: acquiring network state data at the next moment; calculating a reward value based on the network state data at the current moment and the network state data at the next moment; calculating a standard target value based on the reward value and the network state data; acquiring first model parameters of the first agent; and calculating second model parameters based on the standard target value, the first action evaluation value, and the first model parameters, wherein the second model parameters are used to form a second agent.

[0008] According to an embodiment of this disclosure, the intelligent agent is formed based on an action value function. The step of evaluating and scoring the multiple slice actions based on the network state data at the current moment and the preset first intelligent agent to obtain multiple action evaluation values ​​includes: for a slice action, taking the network state data at the current moment, the slice action, and the first model parameters as input data, and calculating the action evaluation value through the action value function.

[0009] According to embodiments of this disclosure, the network status data includes at least the audio recognition speed, and the acquisition of the current network status data includes: acquiring the audio duration, start time, and time when the recognition result is obtained; and calculating the audio recognition speed based on the audio duration, the start time, and the time when the recognition result is obtained.

[0010] According to an embodiment of this disclosure, the step of calculating the reward value based on the network state data at the current moment and the network state data at the next moment includes: calculating the reward value based on the audio recognition speed at the current moment and the audio recognition speed at the previous moment.

[0011] A second aspect of this disclosure provides a speech recognition method, wherein the speech receiving method is applied to a speech recognition server, the method comprising: receiving an audio slice from a user terminal, the user terminal being the initial sender of the audio slice, the audio slice being obtained by the user terminal by segmenting a collected user speech audio stream into frames using current network status data and a preset first intelligent agent; recognizing the audio slice according to a preset speech recognition logic to obtain speech recognition information; and sending the speech recognition information to the user terminal.

[0012] A third aspect of this disclosure provides a speech recognition method, wherein the method is applied to a speech recognition system, the speech recognition system comprising: a user terminal and a speech recognition server, the method comprising: the user terminal acquiring network status data at a current time; based on the network status data at the current time and a preset first agent, segmenting the acquired user speech audio stream into frames to obtain audio slices, wherein the first agent is formed based on reinforcement learning; the speech recognition server receiving the audio slices from the user terminal; recognizing the audio slices according to a preset speech recognition logic to obtain speech recognition information; and sending the speech recognition information to the user terminal.

[0013] A fourth aspect of this disclosure provides a speech recognition device applied to a user terminal. The device includes: a network status acquisition module for acquiring network status data at the current moment; an audio slicing module for segmenting the acquired user speech audio stream into frames based on the current network status data and a preset first intelligent agent to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; a slice sending module for sending the audio slices to a speech recognition server; and a speech recognition information receiving module for receiving speech recognition information from the speech recognition server.

[0014] According to an embodiment of this disclosure, the audio slicing module is configured to acquire a plurality of preset slice actions; evaluate and score the plurality of slice actions based on the network state data at the current time and the preset first agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segment the user's voice audio stream into frames based on the first slice action.

[0015] According to an embodiment of this disclosure, the device further includes an agent update module, configured to: acquire network state data at the next time step; calculate a reward value based on the network state data at the current time step and the network state data at the next time step; calculate a standard target value based on the reward value and the network state data; acquire first model parameters of the first agent; and calculate second model parameters based on the standard target value, the first action evaluation value, and the first model parameters, wherein the second model parameters are used to form a second agent.

[0016] According to an embodiment of this disclosure, the intelligent agent is formed based on an action value function, and the audio slicing module is used to calculate an action evaluation value for a slice action by taking the current network state data, the slice action, and the first model parameters as input data through the action value function.

[0017] According to an embodiment of this disclosure, the network status data includes at least the audio recognition speed, and the network status acquisition module is used to acquire the audio duration, start time, and recognition result acquisition time; and to calculate the audio recognition speed based on the audio duration, the start time, and the recognition result acquisition time.

[0018] According to an embodiment of this disclosure, the agent update module is used to calculate the reward value based on the audio recognition speed at the current moment and the audio recognition speed at the previous moment.

[0019] A fifth aspect of this disclosure provides a speech recognition device applied to a speech recognition server. The device includes: a slice receiving module for receiving audio slices from a user terminal, the user terminal being the initial sender of the audio slices, the audio slices being obtained by the user terminal through frame segmentation of a collected user speech audio stream using current network status data and a preset first intelligent agent; a speech recognition module for recognizing the audio slices according to preset speech recognition logic to obtain speech recognition information; and a speech recognition information sending module for sending the speech recognition information to the user terminal.

[0020] A sixth aspect of this disclosure provides a speech recognition system, the system comprising: a user terminal and a speech recognition server, wherein the user terminal is configured to acquire network status data at a current time; based on the network status data at the current time and a preset first intelligent agent, segment the acquired user speech audio stream into frames to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; and the speech recognition server is configured to receive the audio slices from the user terminal; recognize the audio slices according to a preset speech recognition logic to obtain speech recognition information; and send the speech recognition information to the user terminal.

[0021] A seventh aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the above-described speech recognition method.

[0022] An eighth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the above-described speech recognition method.

[0023] A ninth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described speech recognition method.

[0024] In the embodiments of this disclosure, reinforcement learning is used to dynamically segment user voice audio, which avoids severe recognition delays caused by audio accumulation when the distance between the client and the server is far and the network conditions are complex. The increased latency due to adjusting the segmentation is basically imperceptible to the user, thereby achieving a near real-time effect, greatly improving the user experience, and achieving the best results with the lowest development effort and data center cost. Attached Figure Description

[0025] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0026] Figure 1A This diagram illustrates an application scenario of the speech recognition method according to an embodiment of the present disclosure.

[0027] Figure 1B A schematic block diagram illustrating a reinforcement learning method according to an embodiment of the present disclosure is shown.

[0028] Figure 1C A schematic diagram illustrating a reinforcement learning process according to an embodiment of the present disclosure is shown.

[0029] Figure 2 A flowchart illustrating a speech recognition method according to an embodiment of the present disclosure is shown schematically.

[0030] Figure 3 A flowchart illustrating an audio framing method according to an embodiment of the present disclosure is shown schematically.

[0031] Figure 4 A flowchart illustrating an agent update method according to an embodiment of the present disclosure is shown schematically.

[0032] Figure 5 A flowchart illustrating a network state data acquisition method according to an embodiment of the present disclosure is shown schematically.

[0033] Figure 6 A flowchart illustrating a data acquisition and model update method according to an embodiment of the present disclosure is shown schematically;

[0034] Figure 7 A flowchart illustrating a speech recognition method according to an embodiment of the present disclosure is shown schematically.

[0035] Figure 8 An interactive flowchart between speech recognition systems according to embodiments of the present disclosure is illustrated schematically;

[0036] Figure 9 An architectural diagram of a speech recognition system according to an embodiment of the present disclosure is illustrated.

[0037] Figure 10 A schematic block diagram of a speech recognition device according to an embodiment of the present disclosure is shown.

[0038] Figure 11 A schematic block diagram of a speech recognition device according to an embodiment of the present disclosure is shown; and

[0039] Figure 12 A block diagram schematically illustrates an electronic device suitable for implementing a speech recognition method according to an embodiment of the present disclosure. Detailed Implementation

[0040] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0041] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0042] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0043] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0044] Before detailing the embodiments of this disclosure, the key technical terms involved in this disclosure will be explained one by one as follows:

[0045] Automatic Speech Recognition (ASR) is a technology that converts sound into text. The principle is to segment the sound into multiple small segments, each called a frame. Framing is generally not a simple cutting operation, but rather uses a moving window function. Frames typically overlap, as shown in Figure 1 (see Chapter 5). Each frame is 25 milliseconds long, with an overlap of 25-10=15 milliseconds between any two frames. This is called framing with a frame length of 25ms and a frame shift of 10ms. In this system, this action is performed on the server side, unlike client-side frame acquisition. After framing, the speech becomes many small segments. However, waveforms have almost no descriptive power in the time domain, so waveform transformation is necessary. A common transformation method is to extract MFCC features. Based on the physiological characteristics of the human ear, each frame waveform is transformed into a multi-dimensional vector, which can be simply understood as containing the content information of that frame of speech. This process is called acoustic feature extraction. At this point, the sound becomes a matrix with 12 rows (assuming the acoustic features are 12-dimensional) and N columns, called the observation sequence, where N is the total number of frames. This sequence is then input into a deep learning model for end-to-end recognition.

[0046] Speech recognition toolkit: also known as ASR SDK, is used to convert speech into text.

[0047] TCP: A connection-based network transport protocol.

[0048] UDP: A message-based network transport protocol.

[0049] Because the voice is generated in real time, data cannot be sent before it is generated. After it is generated, the data must be sent to the server immediately so that the server can receive the data and recognize it as soon as possible. In order to ensure low latency, the audio must be sent in real time. In order to ensure stability, the TCP protocol is used and it is a single-threaded serial transmission. This will result in a slow return of recognition results if the distance between the client and the server is far or if the RTT is long due to other reasons.

[0050] The existing solution segments audio data into fixed 80ms intervals. After the SDK sends out 80ms of audio data, it waits for the request to return. During this time, the user continues speaking. If the 80ms audio data is delayed, for example, taking 160ms to return, then 160ms of audio is waiting to be sent. If the next packet (corresponding to an 80ms audio duration) also takes that long, then by the time it finishes sending, the queue will contain 240ms of audio packets (three audio packets), and so on. A user speaking for 4 seconds will require approximately 4 seconds to get a result, instead of just a single packet delay of 80ms (160ms - 80ms). The key issue here is that the latency is determined by the network and cannot be transmitted concurrently. The latency of transmitting an 80ms audio packet is uncontrollable, sometimes 90ms, sometimes 1 second, leading to completely uncontrollable recognition efficiency.

[0051] To address the technical problems existing in the prior art, embodiments of this disclosure provide a speech recognition method applied to a user terminal. The method includes: acquiring network status data at the current moment; segmenting the collected user speech audio stream into frames based on the network status data at the current moment and a preset first intelligent agent to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; sending the audio slices to a speech recognition server; and receiving speech recognition information from the speech recognition server.

[0052] In the embodiments of this disclosure, reinforcement learning is used to dynamically segment user voice audio, which avoids severe recognition delays caused by audio accumulation when the distance between the client and the server is far and the network conditions are complex. The increased latency due to adjusting the segmentation is basically imperceptible to the user, thereby achieving a near real-time effect, greatly improving the user experience, and achieving the best results with the lowest development effort and data center cost.

[0053] Figure 1A The diagram illustrates an application scenario of the speech recognition method according to an embodiment of the present disclosure.

[0054] like Figure 1A As shown, application scenario 100 according to this embodiment may include terminal devices 101, 102, and 103, network 104, and server 105. Network 104 is used as a medium to provide a communication link between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0055] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0056] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0057] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0058] It should be noted that the speech recognition method provided in this embodiment can be executed by terminal devices 101, 102, 103 or server 105 as needed. Accordingly, the data transmission device provided in this embodiment can generally be set in terminal devices 101, 102, 103 or server 105 as needed.

[0059] It should be understood that Figure 1A The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0060] The following will be based on Figure 1A The described scene, through Figures 1B to 9 The speech recognition method of the disclosed embodiments will be described in detail.

[0061] Figure 1B A schematic block diagram illustrating a reinforcement learning method according to an embodiment of the present disclosure is shown.

[0062] like Figure 1BAs shown, reinforcement learning, simply put, involves an agent (the initiator of actions) initiating an action (denoted as A), which then affects the environment. The environment, upon receiving this action, changes its state and generates a reinforcement signal (reward or punishment) as feedback to the agent. The agent then selects its next action based on the reinforcement signal and the current state of the environment, aiming to increase the probability of receiving a positive reinforcement (reward). The chosen action not only affects the current reinforcement value (feedback value) but also the environment's previous state and the final reinforcement value. Reinforcement learning contains two basic elements: state and action. Performing a certain action in a given state constitutes a strategy. The learner's task is to continuously explore and learn strategies to acquire a good one.

[0063] Specifically, the selection of actions is based on the action value function Q. π The implemented value function is expressed as follows:

[0064] U t =R t +γR t+1 +γ 2 R t+2 +... formula (1)

[0065] Where Y refers to the factor in the interval [0, 1], and R refers to the reward.

[0066] Q π (s t a t )=E[U t |s t a t Equation (2)

[0067] Here, S refers to the set of states, A refers to the set of actions, and π refers to the strategy for performing the actions.

[0068] By combining equations (1) and (2), we can use state S and action A as inputs to the model, and combine them with reward R to calculate different rewards for different actions, select the value with the largest reward, and then specify the corresponding strategy through the action A corresponding to the value with the largest reward.

[0069] Figure 1C The diagram illustrates a reinforcement learning process according to an embodiment of the present disclosure.

[0070] like Figure 1CAs shown, this embodiment demonstrates the reinforcement learning process of a Deep Q-learning Network (DQN). Specifically, DQN is denoted as Q*(s, a; w), where the parameters of the neural network are w, the input is the state s, and the output is the score of all actions. We learn the neural network through rewards, and the neural network's scoring of actions will gradually improve, becoming more and more accurate. Currently observed state st, DQN uses st as input to score all actions, selecting the action with the highest score as at. After the agent executes the action at, the environment changes its state, resulting in a new state s. t+1 The environment will also tell us the reward rt for this step. The reward can be positive, negative, or 0.

[0071] Rewards are the supervisory signals used in reinforcement learning. DQN relies on these rewards for training, especially when a new state s is encountered. t+1 The DQN algorithm scores all actions again, and the agent selects the action with the highest score as the action. t+1 Agent executes a t+1 Afterwards, the environment will update the state s again and give another reward r.

[0072] Then, the process is repeated, with DQN scoring the actions, the agent selecting the action with the highest score to execute, the environment updating the state s, and then giving a reward r, and this process is repeated continuously.

[0073] Figure 2 A flowchart illustrating a speech recognition method according to an embodiment of the present disclosure is shown schematically.

[0074] like Figure 2 As shown, the speech recognition method of this embodiment includes operations S210 to S240, and the speech recognition method can be executed by terminals 101, 102, and 103.

[0075] In operation S210, the network status data at the current moment is obtained.

[0076] According to embodiments of this disclosure, the network status data includes: average bandwidth, identification latency, average round-trip time of the minimum network packet, and audio packet identification time.

[0077] Here, average bandwidth refers to the average bandwidth observed in the current time period, denoted as "bw"; recognition latency refers to the recognition latency in the current time period, denoted as "reg_speed"; average round-trip time of the network minimum packet refers to the average round-trip time of the network minimum packet in the current time period, denoted as "rtt_avg"; and audio packet recognition time refers to the ASR engine recognition time of the audio packet in the current time period (corresponding to the recognition result), denoted as "server_reg_time". The above "average bandwidth, recognition latency, average round-trip time of the network minimum packet, and audio packet recognition time" are collectively denoted as the state "S". t ", status "S" t "This is the input to the reinforcement learning network algorithm. The time period can be a fixed-length segment, which can be set by the developers themselves. For example, the period can be set to 1 second or 2 seconds. When controlling speech segments, data representing the network status within a 1-second or 2-second time period can be referenced. Alternatively, the time period can be a non-fixed-length segment, the length of which can be determined based on..."

[0078] It should be noted that the network status data mentioned above are all collected within a certain time period. Due to the differences in data collection for different network status data, the collection period for each network status data is different; that is, the collection time is the same, but the collection period is different. For example, the detection interval for average bandwidth "bw" and average round-trip time of the minimum network packet "rtt_avg" is 10 seconds, while the detection interval for identification delay "reg_speed" is 1 second.

[0079] In operation S220, based on the network state data at the current moment and the preset first intelligent agent, the collected user voice audio stream is framed to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning.

[0080] Specifically, the agent node observes the environmental state and selects an action according to the action selection strategy to control the audio frame size, obtaining the new state and reward after executing the action. Then, the state, the selected action, the new state, and the reward are stored in the experience pool. The agent is the execution unit of the reinforcement learning network algorithm, implemented using the DQN function.

[0081] The agent is formed based on reinforcement learning algorithms. Classic reinforcement learning algorithms include value function-based methods such as Q-learning and Sarsa, and policy search-based methods such as REUNFORCE and Actor-Critic. Furthermore, variations of reinforcement learning include model-based reinforcement learning, inverse reinforcement learning, and meta-reinforcement learning. In the embodiments of this disclosure, methods such as... Figure 1C The deep neural network algorithm in the model takes slice actions as the input action set A and the current network state data as the input state set S. Based on the parameters w in the current DQN model, it outputs the strategy of slice actions.

[0082] It is understandable that the "first" in the above-mentioned first intelligent agent refers to a moment in a non-initial stage. For other non-initial stages, it can also be expressed as "second", "third", etc.

[0083] In operation S230, the audio slice is sent to the speech recognition server.

[0084] In operation S240, speech recognition information is received from the speech recognition server.

[0085] The speech is recognized by a dedicated speech recognition server, which then returns the recognition result, which is usually in the form of text.

[0086] In the embodiments of this disclosure, reinforcement learning is used to dynamically segment user voice audio, which avoids severe recognition delays caused by audio accumulation when the distance between the client and the server is far and the network conditions are complex. The increased latency due to adjusting the segmentation is basically imperceptible to the user, thereby achieving a near real-time effect, greatly improving the user experience, and achieving the best results with the lowest development effort and data center cost.

[0087] Figure 3 A flowchart illustrating an audio framing method according to an embodiment of the present disclosure is shown schematically.

[0088] like Figure 3 As shown, the audio framing method of this embodiment includes operations S310 to S330, which can at least partially perform the above-mentioned operation S220.

[0089] In operation S310, multiple preset slicing actions are obtained.

[0090] Slicing refers to cutting audio into segments of predetermined length.

[0091] For example, actions are used to control the size of audio cuts, and are expressed as functions, as shown below:

[0092] SplitAudio(x) = 50 + 10x (Equation 3)

[0093] SplitAudio(x) is a discrete function, with x ranging from 0 to 45. The value of SplitAudio represents the duration of the original audio segment. For example, when x is 1, its value is 60, meaning the audio data packet is sent to the server for recognition after the user speaks for 60ms. Here, we have set 46 actions, each corresponding to a different frame length (time unit).

[0094] In operation S320, based on the network state data at the current moment and the preset first agent, the multiple slice actions are evaluated and scored to obtain multiple action evaluation values. The multiple action evaluation values ​​include at least a first action evaluation value, which is the highest score among the multiple action evaluation values. The first action evaluation value corresponds to the first slice action.

[0095] According to an embodiment of this disclosure, the intelligent agent is formed based on an action value function. The step of evaluating and scoring the multiple slice actions based on the network state data at the current moment and the preset first intelligent agent to obtain multiple action evaluation values ​​includes: for a slice action, taking the network state data at the current moment, the slice action, and the first model parameters as input data, and calculating the action evaluation value through the action value function.

[0096] In combination with the above Figure 1B and Figure 1C Based on the principles of equations (1) and (2) above, by taking the network state data and slice action at the current moment as input parameters, the score of the slice action is obtained from the parameter w of the current first agent.

[0097] Specifically, observe the current state s t Input state s t Perform a calculation using DQN and output the result for action a. t The scores are assigned, and the highest score 'a' is identified. t .

[0098] In operation S330, the user's voice audio stream is framed based on the first slicing action.

[0099] Based on the cutting length of the first slicing action, the acquired audio stream is cut into segments to complete frame segmentation.

[0100] It should be noted that operations S310 to S330 above need to be executed during the non-initialization phase, meaning that the parameter w in the agent has already been calculated based on historical data, rather than being a random number generated during some initialization process. The iteration process of w in the agent during use is as follows:

[0101] Figure 4 A flowchart illustrating an agent update method according to an embodiment of the present disclosure is shown schematically.

[0102] like Figure 4 As shown, the agent update method in this embodiment includes operations S410 to S450. Operations S410 to S450 are executed after operation S250.

[0103] In operation S410, network status data for the next moment is obtained.

[0104] Since the agent has already performed action a t At this point, the environment will update its state s to S. t+1 Therefore, the network state data for the next moment can be obtained.

[0105] In operation S420, the reward value is calculated based on the network state data at the current moment and the network state data at the next moment.

[0106] Of course, after obtaining network state data at different times, the reward value r can be calculated based on the differences between the network state data at different times. t .

[0107] According to an embodiment of this disclosure, the step of calculating the reward value based on the network state data at the current moment and the network state data at the next moment includes: calculating the reward value based on the audio recognition speed at the current moment and the audio recognition speed at the previous moment.

[0108] Specifically, the reward value is calculated as follows:

[0109] reward=α(reg_speed-reg_speed_last+β) Formula (4)

[0110] Here, reward is the reward r, reg_speed is the audio recognition speed in the most recent time period, and reg_speed_last is the recognition speed in the previous time period.

[0111] α is the gain coefficient. The reward can be positive or negative. A positive reward indicates that the current framing strategy matches the previous strategy of the current network; a negative reward indicates a poor match between the current framing strategy and the previous strategy of the current network, suggesting potential network congestion and requiring adjustment of the frame length. β > 0 is a hyperparameter set to prevent the reward from not increasing even when the network is not congested.

[0112] In operation S430, a standard target value is calculated based on the reward value and the network state data.

[0113] Specifically, with state S t+1 and reward r t We can then use this formula to calculate the standard target value, which can be understood as the labeled value in supervised learning, that is, the correct target value, denoted as y. t The specific calculation method for the standard target value is as follows:

[0114]

[0115] Where γ refers to the factor in the interval [0, 1], and yt is obtained through reward r. t Calculated based on the maximum motion evaluation value.

[0116] In operation S440, the first model parameters of the first agent are obtained.

[0117] The first model parameter is the model parameter in the first intelligent agent. The agent iterates continuously based on external network state data. Therefore, the first model parameter in the first intelligent agent can be regarded as the model parameter that has not been updated at the current moment, denoted as w. t

[0118] In operation S450, based on the standard target value, the first action evaluation value, and the first model parameters, second model parameters are calculated, wherein the second model parameters are used to form a second intelligent agent.

[0119] Accordingly, the second model parameter refers to the model parameter for updating the agent at the current time, denoted as w. t+1 .

[0120] Specifically, in conjunction with the above operation S320, the evaluation value of this first action is the highest value, a. t , here written as q t The calculation method for the second model parameters is as follows:

[0121] Loss=(y t -q t Equation (6)

[0122] w t+1 =w t -η(y t -q t )d t Equation (7)

[0123] The gradient d is obtained by taking the derivative of DQN using backpropagation. t Specifically, Tensorflow and PyTorch can be used, both of which can automatically calculate gradients. η is a hyperparameter, referring to the learning rate.

[0124] The variable reg_speed that needs to be used to calculate the reward in equation (4) above is as follows: Figure 5 As shown:

[0125] Figure 5 A flowchart illustrating a method for acquiring network state data according to an embodiment of this disclosure is shown schematically.

[0126] like Figure 5 As shown, the agent update method of this embodiment includes operations S510 to S520. Operations S510 to S520 can at least partially perform the above-described operation S210.

[0127] According to embodiments of this disclosure, the network status data includes at least the audio recognition speed.

[0128] In operating the S510, the audio duration, start time, and recognition result acquisition time are obtained.

[0129] In operation S520, the audio recognition speed is calculated based on the audio duration, the start time, and the time when the recognition result is obtained.

[0130] Specifically, in the scenarios applied in the embodiments of this disclosure, the reward is reflected in the recognition response speed, where the recognition response speed is denoted as reg_speed, which refers to the difference between the end-to-end audio duration (denoted as audio_len) and its start time (denoted as audio_gen_time) and the time to obtain the recognition result (denoted as audio_reg_res_time). The calculation formula is as follows:

[0131]

[0132] Figure 6 A flowchart illustrating a data acquisition and model update method according to an embodiment of the present disclosure is shown schematically.

[0133] like Figure 6 As shown, the data acquisition and model update method includes operations S610 to S640.

[0134] In operation S610, DQN is randomly initialized.

[0135] During operation of S620, no observation data was acquired at time T0.

[0136] That is, there is no observation data at time T0.

[0137] In operation S630, at time T1, reg_speed and server_reg_time are calculated based on the observed recognition results, and the bw and rtt_avg of the original audio data at that time are found in the historical data. The action is then calculated and selected using reg_speed, server_reg_time, bw and rtt_avg.

[0138] The reward value cannot be calculated at this point. However, the expected value of each action can be calculated based on the current network parameters. This involves substituting the frame duration (SplitAudio(a)) corresponding to the 46 actions into the Q* function, finding the maximum value and the action corresponding to the maximum value, and then selecting that action.

[0139] In operation S640, at time T2, the second identification result is received, and the model parameters are calculated and updated using the standard values.

[0140] When performing audio slicing processing at a later time, simply repeat operations S630 to S640. Of course, the corresponding time points need to be adjusted during the repeated execution.

[0141] Figure 7 A flowchart illustrating a speech recognition method according to an embodiment of the present disclosure is shown schematically.

[0142] like Figure 7 As shown, the speech recognition method includes operations S710 to S730. Operations S710 to S730 can be executed by server 105.

[0143] In operation S710, an audio slice is received from a user terminal, which is the initial sender of the audio slice. The audio slice is obtained by the user terminal by dividing the collected user voice audio stream into frames using the network status data at the current moment and a preset first intelligent agent model.

[0144] In operation S720, the audio slice is recognized according to the preset speech recognition logic to obtain speech recognition information.

[0145] The preset speech recognition logic is the ASR engine, which can directly recognize the received speech and convert it into text.

[0146] In operation S730, the voice recognition information is sent to the user terminal.

[0147] In the embodiments of this disclosure, reinforcement learning is used to dynamically segment user voice audio, which avoids severe recognition delays caused by audio accumulation when the distance between the client and the server is far and the network conditions are complex. The increased latency due to adjusting the segmentation is basically imperceptible to the user, thereby achieving a near real-time effect, greatly improving the user experience, and achieving the best results with the lowest development effort and data center cost.

[0148] Figure 8 A schematic diagram illustrating the data interaction flowchart within a speech recognition system according to an embodiment of the present disclosure is provided.

[0149] like Figure 8 As shown, the microphone collects user voice, sends the collected audio stream to the ASR-SDK for audio slicing, sends the sliced ​​data to the load balancing server, the load balancing server transmits it to the internal and external network isolation server, and then the internal and external network isolation server transmits it to the ASR engine server (speech recognition server).

[0150] After the ASR engine server (speech recognition server) recognizes the content of the audio segment, the text is returned to the internal and external network isolation server. The internal and external network isolation server then directly returns the recognized text to the corresponding client APP on the user terminal. This completes the remote speech recognition.

[0151] Figure 9 An architectural diagram of a speech recognition system according to an embodiment of the present disclosure is illustrated.

[0152] like Figure 9 As shown, after the audio stream is preprocessed by the preprocessing thread, the data sending thread controls the slice size of the audio stream and sends the sliced ​​audio to the ASR server. The probe thread is used to periodically acquire network status data and provide feedback.

[0153] Based on the above speech recognition method, this disclosure also provides a speech recognition device. The following will be combined with... Figure 10 and Figure 11 The device is described in detail.

[0154] Figure 10 A schematic block diagram of a speech recognition device according to an embodiment of the present disclosure is shown.

[0155] like Figure 10As shown, the voice recognition device 1000 of this embodiment includes a network status acquisition module 1010, an audio slicing module 1020, a slice sending module 1030, and a voice recognition information receiving module 1040. This voice recognition device 1000 is applied in terminal devices 101, 102, and 103.

[0156] The network status acquisition module 1010 is used to acquire network status data at the current moment. In one embodiment, the network status acquisition module 1010 can be used to perform the operation S210 described above, which will not be repeated here.

[0157] The audio slicing module 1020 is used to segment the collected user voice audio stream into frames based on the network state data at the current moment and a preset first intelligent agent, thereby obtaining audio slices. The first intelligent agent is formed based on reinforcement learning. In one embodiment, the audio slicing module 1020 can be used to perform the operation S220 described above, which will not be repeated here.

[0158] The slice sending module 1030 is used to send the audio slice to the speech recognition server. In one embodiment, the slice sending module 1030 can be used to perform the operation S230 described above, which will not be repeated here.

[0159] The speech recognition information receiving module 1040 is used to receive speech recognition information from the speech recognition server. In one embodiment, the speech recognition information receiving module 1040 can be used to perform the operation S240 described above, which will not be repeated here.

[0160] In the embodiments of this disclosure, reinforcement learning is used to dynamically segment user voice audio, which avoids severe recognition delays caused by audio accumulation when the distance between the client and the server is far and the network conditions are complex. The increased latency due to adjusting the segmentation is basically imperceptible to the user, thereby achieving a near real-time effect, greatly improving the user experience, and achieving the best results with the lowest development effort and data center cost.

[0161] According to an embodiment of this disclosure, the audio slicing module is configured to acquire a plurality of preset slice actions; evaluate and score the plurality of slice actions based on the network state data at the current time and the preset first agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segment the user's voice audio stream into frames based on the first slice action.

[0162] According to an embodiment of this disclosure, the device further includes an agent update module, configured to: acquire network state data at the next time step; calculate a reward value based on the network state data at the current time step and the network state data at the next time step; calculate a standard target value based on the reward value and the network state data; acquire first model parameters of the first agent; and calculate second model parameters based on the standard target value, the first action evaluation value, and the first model parameters, wherein the second model parameters are used to form a second agent.

[0163] According to an embodiment of this disclosure, the intelligent agent is formed based on an action value function, and the audio slicing module is used to calculate an action evaluation value for a slice action by taking the current network state data, the slice action, and the first model parameters as input data through the action value function.

[0164] According to an embodiment of this disclosure, the network status data includes at least the audio recognition speed, and the network status acquisition module is used to acquire the audio duration, start time, and recognition result acquisition time; and to calculate the audio recognition speed based on the audio duration, the start time, and the recognition result acquisition time.

[0165] According to an embodiment of this disclosure, the agent update module is used to calculate the reward value based on the audio recognition speed at the current moment and the audio recognition speed at the previous moment.

[0166] According to embodiments of this disclosure, any multiple modules among the network status acquisition module 1010, audio slicing module 1020, slice sending module 1030, and voice recognition information receiving module 1040 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the network status acquisition module 1010, audio slicing module 1020, slice sending module 1030, and voice recognition information receiving module 1040 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the network status acquisition module 1010, audio slicing module 1020, slice sending module 1030, and speech recognition information receiving module 1040 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0167] Figure 11 A schematic block diagram of a speech recognition device according to an embodiment of the present disclosure is shown.

[0168] like Figure 11 As shown, the speech recognition device 1100 of this embodiment includes a slice receiving module 1110, a speech recognition module 1120, and a speech recognition information sending module 1130. This speech recognition device 1100 is used in a server 105.

[0169] The receiving module 1110 is used to receive audio slices from a user terminal, which is the initial sender of the audio slices. The audio slices are obtained by the user terminal by dividing the collected user voice audio stream into frames using the network status data at the current moment and a preset first intelligent agent. In one embodiment, the receiving module 1110 can be used to perform the operation S710 described above, which will not be repeated here.

[0170] The speech recognition module 1120 is used to recognize the audio slices according to preset speech recognition logic to obtain speech recognition information. In one embodiment, the speech recognition module 1120 can be used to perform the operation S720 described above, which will not be repeated here.

[0171] The voice recognition information sending module 1130 is used to send the voice recognition information to the user terminal. In one embodiment, the voice recognition information sending module 1130 can be used to perform the operation S730 described above, which will not be repeated here.

[0172] In the embodiments of this disclosure, reinforcement learning is used to dynamically segment user voice audio, which avoids severe recognition delays caused by audio accumulation when the distance between the client and the server is far and the network conditions are complex. The increased latency due to adjusting the segmentation is basically imperceptible to the user, thereby achieving a near real-time effect, greatly improving the user experience, and achieving the best results with the lowest development effort and data center cost.

[0173] According to embodiments of this disclosure, any plurality of modules among the slice receiving module 1110, the voice recognition module 1120, and the voice recognition information sending module 1130 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the slice receiving module 1110, the voice recognition module 1120, and the voice recognition information sending module 1130 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the slice receiving module 1110, the speech recognition module 1120, and the speech recognition information sending module 1130 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0174] Figure 12 A block diagram schematically illustrates an electronic device suitable for implementing a speech recognition method according to an embodiment of the present disclosure.

[0175] like Figure 12As shown, an electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0176] RAM 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Processor 1201 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1202 and / or RAM 1203. It should be noted that the programs may also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0177] According to embodiments of this disclosure, the electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0178] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0179] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1202 and / or RAM 1203 and / or one or more memories other than ROM 1202 and RAM 1203 described above.

[0180] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0181] When the computer program is executed by the processor 1201, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0182] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0183] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0184] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0186] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0187] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A speech recognition method, the method being applied to a user terminal, the method comprising: Obtain the network status data at the current moment; Based on the network state data at the current moment and the preset first intelligent agent, the collected user voice audio stream is framed to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; The audio slice is sent to the speech recognition server; and Receive speech recognition information from the speech recognition server; The step of segmenting the collected user voice audio stream into frames based on the current network state data and a preset first intelligent agent to obtain audio slices includes: acquiring a preset plurality of slice actions; evaluating and scoring the plurality of slice actions based on the current network state data and the preset first intelligent agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segmenting the user voice audio stream into frames based on the first slice action. The slicing action refers to the length of the audio segment cut.

2. The method according to claim 1, wherein, After receiving the speech recognition information from the speech recognition server, the method further includes: Obtain network status data for the next time step; Calculate the reward value based on the network status data at the current moment and the network status data at the next moment; Calculate the standard target value based on the reward value and the network status data; Obtain the first model parameters of the first agent; and Based on the standard target value, the first action evaluation value, and the first model parameters, the second model parameters are calculated, wherein the second model parameters are used to form the second intelligent agent.

3. The method according to claim 2, wherein, The intelligent agent is formed based on the action value function. Based on the network state data at the current moment and the preset first intelligent agent, the multiple slice actions are evaluated and scored to obtain multiple action evaluation values, including: For a given slice action, the current network state data, the slice action, and the first model parameters are used as input data, and the action evaluation value is calculated using the action value function.

4. The method according to claim 2 or 3, wherein, The network status data includes at least the audio recognition speed. The process of obtaining the network status data at the current moment includes: Obtain the audio duration, start time, and time when the recognition result is received; The audio recognition speed is calculated based on the audio duration, the start time, and the time it takes to obtain the recognition result.

5. The method according to claim 4, wherein, The calculation of the reward value based on the network state data at the current moment and the network state data at the next moment includes: The reward value is calculated based on the audio recognition speed at the current moment and the audio recognition speed at the previous moment.

6. A speech recognition method, wherein, The voice receiving method is applied to a voice recognition server, and the method includes: The system receives audio slices from a user terminal, which is the initial sender of the audio slices. The audio slices are obtained by the user terminal by dividing the collected user voice audio stream into frames using the network status data at the current moment and a preset first intelligent agent. According to the preset speech recognition logic, the audio slices are recognized to obtain speech recognition information; and The voice recognition information is sent to the user terminal; The audio slices are obtained by the user terminal dividing the collected user voice audio stream into frames based on the current network status data and a preset first intelligent agent. This process includes: acquiring multiple preset slice actions; evaluating and scoring the multiple slice actions based on the current network status data and the preset first intelligent agent to obtain multiple action evaluation values, wherein the multiple action evaluation values ​​include at least a first action evaluation value, which is the highest score among the multiple action evaluation values ​​and corresponds to a first slice action; and dividing the user voice audio stream into frames based on the first slice action. The slice action refers to the length of the audio segment.

7. A speech recognition method, wherein, The method is applied to a speech recognition system, the speech recognition system comprising: The method includes a user terminal and a speech recognition server, and the method comprises: The user terminal obtains the network status data at the current moment; Based on the network state data at the current moment and the preset first intelligent agent, the collected user voice audio stream is framed to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; The speech recognition server receives audio slices from the user terminal; According to the preset speech recognition logic, the audio slices are recognized to obtain speech recognition information; and The voice recognition information is sent to the user terminal; The step of segmenting the collected user voice audio stream into frames based on the current network state data and a preset first intelligent agent to obtain audio slices includes: acquiring a preset plurality of slice actions; evaluating and scoring the plurality of slice actions based on the current network state data and the preset first intelligent agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segmenting the user voice audio stream into frames based on the first slice action. The slicing action refers to the length of the audio segment cut.

8. A voice recognition device, the device being applied to a user terminal, the device comprising: The network status acquisition module is used to acquire network status data at the current moment. The audio slicing module is used to divide the collected user voice audio stream into frames based on the network state data at the current moment and the preset first intelligent agent to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; A slice sending module is used to send the audio slice to a speech recognition server; and A speech recognition information receiving module is used to receive speech recognition information from the speech recognition server; The step of segmenting the collected user voice audio stream into frames based on the current network state data and a preset first intelligent agent to obtain audio slices includes: acquiring a preset plurality of slice actions; evaluating and scoring the plurality of slice actions based on the current network state data and the preset first intelligent agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segmenting the user voice audio stream into frames based on the first slice action. The slicing action refers to the length of the audio segment cut.

9. A speech recognition device, the device being applied to a speech recognition server, the device comprising: The audio slice receiving module is used to receive audio slices from a user terminal, where the user terminal is the initial sender of the audio slice. The audio slice is obtained by the user terminal by dividing the collected user voice audio stream into frames using the network status data at the current moment and a preset first intelligent agent. The speech recognition module is used to recognize the audio slices according to preset speech recognition logic to obtain speech recognition information; as well as A voice recognition information sending module is used to send the voice recognition information to the user terminal; The audio slices are obtained by the user terminal dividing the collected user voice audio stream into frames based on the current network status data and a preset first intelligent agent. This process includes: acquiring multiple preset slice actions; evaluating and scoring the multiple slice actions based on the current network status data and the preset first intelligent agent to obtain multiple action evaluation values, wherein the multiple action evaluation values ​​include at least a first action evaluation value, which is the highest score among the multiple action evaluation values ​​and corresponds to a first slice action; and dividing the user voice audio stream into frames based on the first slice action. The slice action refers to the length of the audio segment.

10. A speech recognition system, the system comprising: User terminal and speech recognition server, among which... The user terminal is configured to acquire network status data at the current moment; based on the network status data at the current moment and a preset first intelligent agent, to segment the collected user voice audio stream into frames to obtain audio slices, wherein the first intelligent agent is formed based on reinforcement learning; and The speech recognition server is configured to receive audio slices from the user terminal; recognize the audio slices according to a preset speech recognition logic to obtain speech recognition information; and send the speech recognition information to the user terminal. The step of segmenting the collected user voice audio stream into frames based on the current network state data and a preset first intelligent agent to obtain audio slices includes: acquiring a preset plurality of slice actions; evaluating and scoring the plurality of slice actions based on the current network state data and the preset first intelligent agent to obtain a plurality of action evaluation values, wherein the plurality of action evaluation values ​​include at least a first action evaluation value, the first action evaluation value being the highest score among the plurality of action evaluation values, and the first action evaluation value corresponding to a first slice action; and segmenting the user voice audio stream into frames based on the first slice action. The slicing action refers to the length of the audio segment cut.

11. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 7.

12. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images