Efficient transmission method and device of audio data and computer equipment

Abstract

The invention relates to an efficient transmission method and device of audio data and computer equipment. The method comprises the following steps: acquiring current sound data information of a user question through a sound acquisition unit, and performing audio coding processing on the current sound data information to obtain a sound data stream; sending the sound data stream to a server through a network sending program, and returning to execute the above steps until a sound data acquisition completion identifier is identified, and sending a sound acquisition completion message to the server; receiving the reply information and performing audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information; and carrying out sound broadcast processing on the reply sound data through a sound broadcast unit, returning to execute the above steps, and stopping question reply processing until a reply completion message fed back by the server is received. By adopting the method, the audio transmission efficiency between the equipment of the traditional processor and the server can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio data transmission, and in particular to an efficient audio data transmission method, device and computer equipment. Background Art

[0002] When the AI ​​big model provides intelligent voice Q&A functions for embedded devices, it involves the transmission of audio data between the device and the server. When the big model responds to the user's Q&A, it is greatly affected by the audio data transmission efficiency between the device and the server. That is, the higher the transmission efficiency, the faster the Q&A response efficiency, and the lower the transmission efficiency, the slower the Q&A response efficiency. Therefore, how to improve the audio transmission efficiency between the device and the server is the current research focus.

[0003] There are three traditional ways to transmit audio data between devices and servers: 1. File transmission. 2. Slice transmission. 3. Streaming transmission. Among the above three transmission methods, streaming transmission is more efficient, but when streaming transmission is applied and 1. the device provides AI intelligent functions, the requirements for the device processor are high and the cost is high. When the first and second methods are applied, the low-cost processor used to complete AI intelligent voice interaction through the network has a large delay. As a result, the audio transmission efficiency between the device and the server of the traditional processor is low. Summary of the invention

[0004] Based on this, it is necessary to provide an efficient transmission method, device, computer equipment, computer-readable storage medium and computer program product for audio data in order to solve the above technical problems.

[0005] In a first aspect, the present application provides an efficient transmission method for audio data, comprising:

[0006] The current sound data information of the user's question is collected through the sound collection unit, and the current sound data information is subjected to audio coding processing to obtain a sound data stream;

[0007] The sound data stream is sent to the server through a network sending program, and the step of collecting the sound data information of the current sound input is returned to execute, until the sound data information contains a sound data collection completion mark, and then a sound collection completion message is sent to the server;

[0008] Receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information;

[0009] The reply sound data is processed by sound broadcasting through a sound broadcasting unit, and the step of receiving the reply information fed back by the server is returned to be executed, until the reply completion message fed back by the server is received, and then the question reply processing is stopped.

[0010] Optionally, performing audio encoding processing on the current sound data information to obtain a sound data stream includes:

[0011] Extracting audio data and audio sequence from the current sound data information;

[0012] The audio data is sequentially encoded through an audio encoder according to the audio sequence to generate a sound data stream corresponding to each time point.

[0013] Optionally, the sending of the sound data stream to the server through a network sending program includes:

[0014] When a sound data stream at each time point is generated, the sound data stream is directly sent to the server through a network sending program.

[0015] Optionally, the performing sound broadcast processing on the reply sound data by a sound broadcast unit includes:

[0016] When each reply sound data corresponding to a reply message is obtained through decoding, the sound broadcasting unit directly performs sound broadcasting processing.

[0017] In a second aspect, the present application provides an efficient transmission method for audio data, comprising:

[0018] The sound data stream transmitted by the receiving device is converted into text by the receiving device to obtain the question text content corresponding to the sound data stream; the sound data stream is obtained by the device performing audio coding processing on the current sound data information;

[0019] Upon receiving the sound collection completion message transmitted by the device, the semantic recognition network is used to identify the user question semantics corresponding to each question text content collected, and the artificial intelligence answering unit is used to generate the reply text content corresponding to the user question semantics;

[0020] According to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to sequence audio conversion processing through an audio decoder to obtain each reply audio data, and after each sub-reply text content completes the audio conversion processing, the reply information containing each reply audio data is respectively sent to the device through a network sending program;

[0021] After completing the sending process of the last reply message, a preset reply completion message is sent to the device.

[0022] Optionally, the step of identifying the user question semantics corresponding to each of the collected question text contents through a semantic recognition network includes:

[0023] Arrange the question text contents according to the conversion time point order of the question text contents to obtain comprehensive question text contents;

[0024] The comprehensive question text content is subjected to semantic recognition processing through a semantic recognition network to obtain semantic information corresponding to the comprehensive question text content, and question semantic information in the semantic information is extracted as user question semantics.

[0025] Optionally, according to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to sequence audio conversion processing through an audio decoder to obtain each reply audio data, including:

[0026] Identify a text sequence in the reply text content, and split the text sequence according to preset text reply units to obtain a target text sequence of each sub-reply text content;

[0027] According to the order of the target text sequence, each sub-reply text content is respectively processed by an audio decoder to perform sequence audio conversion processing to obtain reply audio data corresponding to each sub-reply text content.

[0028] In a third aspect, the present application also provides an efficient transmission system for audio data, the system comprising a device and a server:

[0029] The device includes a sound collection unit and a sound broadcasting unit, and the device is remotely connected to the server;

[0030] The device is used to collect the current sound data information of the user's question through the sound collection unit, and perform audio encoding processing on the current sound data information to obtain a sound data stream; send the sound data stream to the server through a network sending program, and return to execute the step of collecting the sound data information of the current sound input, until the sound data information contains a sound data collection completion mark, and then send a sound collection completion message to the server;

[0031] The server is used to receive the sound data stream transmitted by the device, and perform text conversion processing on the sound data stream to obtain the question text content corresponding to the sound data stream; when receiving the sound collection completion message transmitted by the device, for each collected question text content, the user question semantics corresponding to each question text content are identified through a semantic recognition network, and a reply text content corresponding to the user question semantics is generated through an artificial intelligence answering unit; according to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to sequence audio conversion processing through an audio decoder to obtain each reply audio data, and after each sub-reply text content completes the audio conversion processing, the reply information containing each reply audio data is respectively sent to the device through a network sending program; after completing the sending process of the last reply message, a preset reply completion message is sent to the device;

[0032] The device is used to receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information; perform sound broadcast processing on the reply sound data through a sound broadcast unit, and return to execute the step of receiving the reply information fed back by the server until the reply completion message fed back by the server is received, and then stop the question reply processing.

[0033] In a fourth aspect, the present application also provides an efficient transmission device for audio data, comprising:

[0034] A collection module, used to collect the current sound data information of the user's question through a sound collection unit, and perform audio coding processing on the current sound data information to obtain a sound data stream;

[0035] A first sending module, configured to send the sound data stream to the server through a network sending program, and return to execute the step of collecting the sound data information of the current sound input, until the sound data information contains a sound data collection completion mark, and then send a sound collection completion message to the server;

[0036] The first receiving module is used to receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information;

[0037] The broadcast module is used to perform sound broadcast processing on the reply sound data through the sound broadcast unit, and return to execute the step of receiving the reply information feedback from the server until the reply completion message feedback from the server is received, and then stop the question reply processing.

[0038] Optionally, the acquisition module is specifically used to:

[0039] Extracting audio data and audio sequence from the current sound data information;

[0040] The audio data is sequentially encoded through an audio encoder according to the audio sequence to generate a sound data stream corresponding to each time point.

[0041] Optionally, the first sending module is specifically configured to:

[0042] When a sound data stream at each time point is generated, the sound data stream is directly sent to the server through a network sending program.

[0043] Optionally, the broadcast module is specifically used to:

[0044] When each reply sound data corresponding to a reply message is obtained through decoding, the sound broadcasting unit directly performs sound broadcasting processing.

[0045] In a fifth aspect, the present application also provides an efficient transmission device for audio data, including:

[0046] The second receiving module is used to receive the sound data stream transmitted by the device, and convert the sound data stream into text to obtain the question text content corresponding to the sound data stream; the sound data stream is obtained by the device performing audio coding processing on the current sound data information;

[0047] A generation module is used to, upon receiving a sound collection completion message transmitted by the device, identify the user question semantics corresponding to each of the collected question text contents through a semantic recognition network, and generate a reply text content corresponding to the user question semantics through an artificial intelligence answering unit;

[0048] A second sending module is used to perform a sequence audio conversion process on each sub-reply text content in the reply text content through an audio decoder according to the text sequence of the reply text content to obtain each reply audio data, and after each sub-reply text content completes the audio conversion process, respectively send the reply information containing each reply audio data to the device through a network sending program;

[0049] The reply module is used to send a preset reply completion message to the device after completing the sending process of the last reply message.

[0050] Optionally, the generating module is specifically used for:

[0051] Arrange the question text contents according to the conversion time point order of the question text contents to obtain comprehensive question text contents;

[0052] The comprehensive question text content is subjected to semantic recognition processing through a semantic recognition network to obtain semantic information corresponding to the comprehensive question text content, and question semantic information in the semantic information is extracted as user question semantics.

[0053] Optionally, the second sending module is specifically configured to:

[0054] Identify a text sequence in the reply text content, and split the text sequence according to preset text reply units to obtain a target text sequence of each sub-reply text content;

[0055] According to the order of the target text sequence, each sub-reply text content is respectively processed by an audio decoder to perform sequence audio conversion processing to obtain reply audio data corresponding to each sub-reply text content.

[0056] In a sixth aspect, the present application provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the methods in the first aspect are implemented.

[0057] In a seventh aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0058] In an eighth aspect, the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0059] The above-mentioned efficient transmission method, device and computer equipment for audio data collect the current sound data information of the user's question through a sound collection unit, and perform audio encoding processing on the current sound data information to obtain a sound data stream; send the sound data stream to the server through a network sending program, and return to execute the step of collecting the sound data information of the current sound input until the sound data information contains a sound data collection completion mark, and then send a sound collection completion message to the server; receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain the reply sound data corresponding to the reply information; perform sound broadcast processing on the reply sound data through a sound broadcast unit, and return to execute the step of receiving the reply information fed back by the server until the reply completion message fed back by the server is received, and then the question reply processing is stopped. When the device and the server are interacting with each other through audio data, this solution will perform audio encoding on the device side to obtain a sound data stream, and then send each encoded sound data stream to the server side instantly through the network sending program by encoding and sending at the same time, thereby reducing the time it takes for the server to obtain all the sound data information, improving the efficiency of the server in obtaining each sound data stream, and improving the efficiency of audio data transmission from the device side to the server side. Then, this solution will send the generated reply information to the device strategy through the server side, so that it can be sent and broadcasted through the sound broadcast unit on the device side, which not only improves the efficiency of audio data transmission from the server side to the device side, but also improves the efficiency of the device in replying to user questions and improves the user experience. Compared with the streaming transmission method, this solution does not require high requirements for the device's processor. Instead, it puts all processing tasks on the server side for execution through real-time sound data transmission, and the device is only responsible for sound data collection and reply data broadcasting. It not only greatly reduces the processing function requirements of the device, but also improves the generation efficiency and accuracy of reply data through comprehensive processing on the server. In addition, the real-time data transmission and real-time reply data broadcasting designed by this solution greatly reduce the audio transmission time, thereby improving the audio transmission efficiency between the device and the server for traditional processors. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1is a system structure diagram of an efficient transmission system for audio data in one embodiment;

[0062] Figure 2 is a flow chart of an efficient audio data transmission method (device side) in one embodiment;

[0063] Figure 3 A schematic diagram of a flow chart of an efficient audio data transmission method (server side) in one embodiment;

[0064] Figure 4 A flowchart of an example of efficient transmission interaction of audio data in one embodiment;

[0065] Figure 5 is a structural block diagram of an apparatus for efficient transmission (device side) of audio data in one embodiment;

[0066] Figure 6 is a structural block diagram of an apparatus for efficient transmission (server side) of audio data in one embodiment;

[0067] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] The efficient transmission method of audio data provided by the embodiment of the present application can be applied to Figure 1The efficient transmission system of audio data shown in the figure. The system includes a device and a server, and the device and the server are remotely connected by a wireless connection. The above-mentioned device can be a terminal. The terminal can be, but is not limited to, various intelligent data collection and broadcasting devices, such as smart speakers, smart watches, smart calendars, smart question-answering machines, smart bracelets and other intelligent devices. The terminal performs audio encoding on each collected sound data information to obtain a sound data stream, and sends each encoded sound data stream to the server side in real time through the network sending program by encoding and sending at the same time, thereby reducing the time for the server to obtain all sound data information, improving the efficiency of the server to obtain each sound data stream, and improving the efficiency of audio data transmission from the device side to the server side. Then, the scheme sends the generated reply information to the device strategy respectively through the server side, so that the sound broadcast unit on the device side can send and broadcast at the same time, which not only improves the efficiency of audio data transmission from the server side to the device side, but also improves the efficiency of the device to reply to user questions and improves the user experience. Compared with the streaming transmission method, this solution does not require high requirements for the device's processor. Instead, it puts all processing tasks on the server side for execution through real-time sound data transmission, and the device is only responsible for sound data collection and reply data broadcasting. It not only greatly reduces the processing function requirements of the device, but also improves the generation efficiency and accuracy of reply data through comprehensive processing on the server. In addition, the real-time data transmission and real-time reply data broadcasting designed by this solution greatly reduce the audio transmission time, thereby improving the audio transmission efficiency between the device and the server for traditional processors.

[0070] In an exemplary embodiment, Figure 2 As shown, a method for efficiently transmitting audio data is provided, and the method is applied to Figure 1 The device shown is described as a terminal, including the following steps S201 to S204. Among them:

[0071] Step S201, collecting the current sound data information of the user's question through the sound collection unit, and performing audio encoding processing on the current sound data information to obtain a sound data stream.

[0072] In this embodiment, the terminal collects the current sound data information of the user's question through the built-in sound collection unit, and performs audio encoding processing on the current sound data information to obtain a sound data stream. Among them, the sound collection unit is a hardware device that can collect sound data, such as a microphone, etc. The audio encoding process is to encode the collected sound into PCM audio data through the encoder RINGBUFFER to obtain a sound data stream. Among them, when encoding, this solution uses the method of encoding while collecting, that is, directly encoding the collected sound data, reducing the waiting time between collecting data and sound encoding, and improving the encoding efficiency.

[0073] Step S202, sending the sound data stream to the server through the network sending program, and returning to execute the step of collecting the sound data information of the current sound input, until the sound data information contains the sound data collection completion mark, and then sending a sound collection completion message to the server.

[0074] In this embodiment, the terminal sends the sound data stream to the server through the network sending program, and returns to execute the sound data information step of collecting the current sound input, until the sound data information contains the sound data collection completion mark, and sends the sound collection completion message to the server. Among them, after completing the collection of all sound data streams, the terminal generates a sound data collection completion mark and sends the mark information to the server. Using this method, it saves the use of device memory and saves the time of copying data in the memory. Among them, the network sending program is an instant sending program preset in the terminal by the staff, and the sending program can send each generated sound data stream directly to the server, thereby avoiding the internal occupancy of the device after storing multiple data streams and then sending them, and the inefficiency of memory copying, improving the sending time of the sound data stream, and reducing the functional requirements of the device.

[0075] Step S203, receiving the reply information fed back by the server, and performing audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information.

[0076] In this embodiment, the terminal receives the reply information fed back by the server, and performs audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information. The audio connection method is to convert the reply audio data in the reply information into sound data that can be broadcast by the sound broadcast unit through the audio decoder.

[0077] Step S204, the reply sound data is processed by the sound broadcast unit, and the step of receiving the reply information fed back by the server is returned to be executed, until the reply completion message fed back by the receiving server is received, and the question reply processing is stopped.

[0078] In this embodiment, the terminal performs voice broadcast processing on the reply voice data through the voice broadcast unit, and returns to the step of receiving the reply information fed back by the server until the reply completion message fed back by the server is received, and then stops the question reply processing. The voice broadcast unit is any broadcast device that can broadcast voice data, such as a speaker, a loudspeaker, a speaker, a headset, etc.

[0079] Based on the above scheme, each collected sound data information is audio-encoded to obtain a sound data stream, and each encoded sound data stream is instantly sent to the server side through the network sending program by encoding and sending at the same time, thereby reducing the time for the server to obtain all sound data information, improving the efficiency of the server to obtain each sound data stream, and improving the efficiency of audio data transmission from the device side to the server side. Then, this scheme sends the generated reply information to the device strategy separately through the server side, so that it is sent and broadcasted through the sound broadcast unit on the device side, which not only improves the efficiency of audio data transmission from the server side to the device side, but also improves the efficiency of the device in replying to user questions, and improves the user experience. Compared with the streaming transmission method, this solution does not require high requirements for the device's processor. Instead, it puts all processing tasks on the server side for execution through real-time sound data transmission, and the device is only responsible for sound data collection and reply data broadcasting. It not only greatly reduces the processing function requirements of the device, but also improves the generation efficiency and accuracy of reply data through comprehensive processing on the server. In addition, the real-time data transmission and real-time reply data broadcasting designed by this solution greatly reduce the audio transmission time, thereby improving the audio transmission efficiency between the device and the server for traditional processors.

[0080] Optionally, the current sound data information is subjected to audio encoding processing to obtain a sound data stream, including: extracting audio data and audio sequence from the current sound data information; sequentially encoding the audio data according to the audio sequence through an audio encoder to generate a sound data stream corresponding to each time point.

[0081] In this embodiment, the terminal extracts the audio data and the audio sequence in the current sound data information. Then, the terminal sequentially encodes the audio data through the audio encoder according to the audio sequence to generate a sound data stream corresponding to each time point. The encoding method of the audio encoder is real-time encoding, that is, the acquired audio data is encoded according to the order of the acquired time points, avoiding the inefficiency of storing the acquired audio data and then calling all the audio data in the storage space for overall decoding.

[0082] Based on the above scheme, variable acquisition side encoding is realized through real-time encoding operation, which improves the efficiency of converting current sound data information into audio data.

[0083] Optionally, the sound data stream is sent to the server through a network sending program, including: when the sound data stream at each time point is generated, the sound data stream is directly sent to the server through a network sending program.

[0084] In this embodiment, when the terminal generates a sound data stream at each time point, it directly sends the sound data stream to the server through the network sending program. That is, this solution uses an instant sending strategy to send each sound data stream to the server in real time in a way of sending while generating.

[0085] Based on the above scheme, the efficiency of sending the sound data stream to the server is improved through instant sending, while the functional requirements of the terminal hardware equipment are reduced, thereby greatly reducing the sending efficiency of the hardware equipment.

[0086] Optionally, the reply sound data is processed by sound broadcasting through a sound broadcasting unit, including: when each reply sound data corresponding to a reply message is obtained through decoding, the reply sound data is directly processed by the sound broadcasting unit.

[0087] In this embodiment, when the terminal decodes and obtains reply sound data corresponding to a reply message, it directly performs sound broadcast processing through the sound broadcast unit. That is, this solution realizes efficient broadcast of reply sound data by real-time sound broadcast processing, decoding and broadcasting at the same time.

[0088] Based on the above scheme, the efficiency of reporting the reply sound data is improved by decoding and reporting simultaneously.

[0089] In an exemplary embodiment, Figure 3 As shown, a method for efficiently transmitting audio data is provided, and the method is applied to Figure 1 The server shown in the figure is described, including the following steps S301 to S304. Among them:

[0090] S301, receiving a sound data stream transmitted by a device, and converting the sound data stream into text to obtain question text content corresponding to the sound data stream.

[0091] The sound data stream is obtained by performing audio encoding processing on the current sound data information by the device.

[0092] In this embodiment, when the server receives each sound data stream, it directly converts the sound data stream into the text content corresponding to the sound data stream through the text conversion network to obtain the question text content corresponding to the sound data stream. That is, this solution avoids the inefficiency of converting text after all sound data streams are accumulated and collected by converting at the same time.

[0093] S302, when receiving the sound collection completion message transmitted by the device, the semantic recognition network is used to identify the user question semantics corresponding to each question text content collected, and the artificial intelligence answering unit is used to generate the reply text content corresponding to the user question semantics.

[0094] In this embodiment, when the server receives the sound collection completion message transmitted by the device, it sorts the collected question text content according to the generation time of each question text content, obtains the complete question text content of the user question, and then identifies the user question semantics corresponding to each question text content through the semantic recognition network, and generates the reply text content corresponding to the user question semantics through the artificial intelligence answer unit. Among them, the artificial intelligence answer unit is an AI large model, and the AI ​​large model and the semantic recognition network are processing programs preset in the server.

[0095] S303, according to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to serial audio conversion processing through an audio decoder to obtain each reply audio data, and after the audio conversion processing of each sub-reply text content is completed, the reply information containing each reply audio data is sent to the device through a network sending program.

[0096] In this embodiment, the server performs a sequence audio conversion process on each sub-reply text content in the reply text content through an audio decoder according to the text sequence of the reply text content to obtain each reply audio data, and after each sub-reply text content completes the audio conversion process, the server sends the reply information containing each reply audio data to the device through a network sending program. The sub-reply text content is the sub-text content obtained after the text reply unit preset by the staff is split into text. The network sending program is the same as the network sending program set in the device.

[0097] S304: After completing the sending process of the last reply message, send a preset reply completion message to the device.

[0098] In this embodiment, after completing the process of sending the last reply message, the server sends a preset reply completion message to the device.

[0099] Based on the above solution, the generated reply information is sent to the device strategy separately, so that it can be sent and broadcasted through the sound broadcast unit on the device side. This not only improves the efficiency of audio data transmission from the server side to the device side, but also improves the efficiency of the device in replying to user questions, thereby improving the user experience.

[0100] Optionally, the collected question text contents are identified through a semantic recognition network to identify the user question semantics corresponding to each question text content, including: arranging each question text content in the order of conversion time points to obtain comprehensive question text content; performing semantic recognition processing on the comprehensive question text content through a semantic recognition network to obtain semantic information corresponding to the comprehensive question text content, and extracting the question semantic information in the semantic information as the user question semantics.

[0101] In this embodiment, the server arranges the text content of each question in the order of the conversion time point of each question text content to obtain the comprehensive question text content. Then, the server performs semantic recognition processing on the comprehensive question text content through the semantic recognition network preset in the server to obtain the semantic information corresponding to the comprehensive question text content, and extracts the question semantic information from the semantic information through the semantic extraction network preset in the server as the user question semantics.

[0102] Based on the above solution, by placing all the processes of text semantic recognition and semantic extraction on the server side, the need for device functions is avoided, the processor cost of the device is reduced, and the efficiency of audio data interaction with different devices is improved.

[0103] Optionally, according to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to serial audio conversion processing through an audio decoder to obtain each reply audio data, including: identifying the text sequence in the reply text content, and splitting the text sequence according to preset text reply units to obtain a target text sequence of each sub-reply text content; according to the order of the target text sequence, each sub-reply text content is subjected to serial audio conversion processing through an audio decoder to obtain reply audio data corresponding to each sub-reply text content.

[0104] In this embodiment, the server identifies the text sequence in the reply text content, and splits the text sequence according to the preset text reply unit to obtain the target text sequence of each sub-reply text content. The text reply unit is a unit preset in the server by the staff, and the size of the unit can be but is not limited to 2 to 5 words.

[0105] Then, the terminal converts each sub-reply text content into a serial audio through the RING BUFFER of the audio decoder in the order of the target text sequence to obtain the reply audio data corresponding to each sub-reply text content.

[0106] Based on the above solution, by splitting the reply information into replies, while ensuring the rationality of the reply content, the problem of waiting time for overall recovery is avoided, thereby improving the reply efficiency of audio data.

[0107] In an exemplary embodiment, a system for efficiently transmitting audio data is provided, wherein the system includes a device and a server: the device includes a sound collection unit and a sound broadcast unit, and the device is remotely connected to the server.

[0108] The device may be, but is not limited to, a smart device including sound collection and sound broadcasting, such as a smart bracelet, a smart watch, smart glasses, smart headphones, a smart speaker, a smart calendar, a smart interactive machine, etc. The remote connection method may be various wireless remote connection methods.

[0109] When the device and the server are transmitting interactive audio data, it includes:

[0110] The current sound data information of the user's question is collected through the sound collection unit, and the current sound data information is subjected to audio encoding processing to obtain a sound data stream; the sound data stream is sent to the server through the network sending program, and the step of collecting the sound data information of the current sound input is returned to execute, until the sound data information contains a sound data collection completion mark, and a sound collection completion message is sent to the server.

[0111] Receive the sound data stream transmitted by the device, and perform text conversion processing on the sound data stream to obtain the question text content corresponding to the sound data stream; when receiving the sound collection completion message transmitted by the device, identify the user question semantics corresponding to each question text content through a semantic recognition network for each collected question text content, and generate a reply text content corresponding to the user question semantics through an artificial intelligence answering unit; according to the text sequence of the reply text content, perform sequence audio conversion processing on each sub-reply text content in the reply text content through an audio decoder to obtain each reply audio data, and after each sub-reply text content completes the audio conversion processing, send the reply information containing each reply audio data to the device through a network sending program; after completing the sending process of the last reply message, send a preset reply completion message to the device;

[0112] Receive the reply information fed back by the server, and perform audio decoding on the reply information to obtain the reply sound data corresponding to the reply information; perform sound broadcast processing on the reply sound data through the sound broadcast unit, and return to execute the step of receiving the reply information fed back by the server, until the reply completion message fed back by the receiving server is received, and then stop the question reply processing.

[0113] This application also provides a network attack protection interaction example for log analysis, such as Figure 4 As shown, the specific processing process includes the following steps:

[0114] Step S401, collecting the current sound data information of the user's question through the sound collection unit, and performing audio coding processing on the current sound data information to obtain a sound data stream.

[0115] Step S402, sending the sound data stream to the server through the network sending program, and sending a sound collection completion message to the server when the sound data information contains a sound data collection completion mark.

[0116] Step S403, converting the sound data stream into text to obtain the question text content corresponding to the sound data stream.

[0117] Step S404, when the sound collection completion message transmitted by the device is received, the semantic recognition network is used to identify the user question semantics corresponding to each question text content collected, and the artificial intelligence answering unit is used to generate the reply text content corresponding to the user question semantics.

[0118] Step S405, according to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to serial audio conversion processing through an audio decoder to obtain each reply audio data, and after the audio conversion processing is completed for each sub-reply text content, the reply information containing each reply audio data is sent to the device through a network sending program; after completing the sending process of the last reply message, a preset reply completion message is sent to the device.

[0119] Step S406, performing audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information, and performing sound broadcast processing on the reply sound data through a sound broadcast unit.

[0120] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiment of the present application also provides an efficient audio data transmission device for implementing the efficient audio data transmission method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in the embodiments of one or more efficient audio data transmission devices provided below can refer to the limitations of the efficient audio data transmission method above, and will not be repeated here.

[0122] In an exemplary embodiment, Figure 5 As shown, a highly efficient transmission device for audio data is provided, comprising: a collection module 510, a first sending module 520, a first receiving module 530 and a broadcasting module 540, wherein:

[0123] The collection module 510 is used to collect the current sound data information of the user's question through the sound collection unit, and perform audio coding processing on the current sound data information to obtain a sound data stream;

[0124] The first sending module 520 is used to send the sound data stream to the server through the network sending program, and return to execute the step of collecting the sound data information of the current sound input until the sound data information contains a sound data collection completion mark, and then send a sound collection completion message to the server;

[0125] The first receiving module 530 is used to receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information;

[0126] The broadcast module 540 is used to perform sound broadcast processing on the reply sound data through a sound broadcast unit, and return to execute the step of receiving the reply information feedback from the server until the reply completion message feedback from the server is received, and then stop the question reply processing.

[0127] Optionally, the acquisition module 510 is specifically used to:

[0128] Extracting audio data and audio sequence from the current sound data information;

[0129] The audio data is sequentially encoded through an audio encoder according to the audio sequence to generate a sound data stream corresponding to each time point.

[0130] Optionally, the first sending module 520 is specifically configured to:

[0131] When a sound data stream at each time point is generated, the sound data stream is directly sent to the server through a network sending program.

[0132] Optionally, the broadcast module 540 is specifically used to:

[0133] When each reply sound data corresponding to a reply message is obtained through decoding, the sound broadcasting unit directly performs sound broadcasting processing.

[0134] Based on the same inventive concept, the embodiment of the present application also provides an efficient audio data transmission device for implementing the efficient audio data transmission method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in the embodiments of one or more efficient audio data transmission devices provided below can refer to the limitations of the efficient audio data transmission method above, and will not be repeated here.

[0135] In an exemplary embodiment, Figure 6 As shown, an efficient transmission device for audio data is provided, comprising: a second receiving module 610, a generating module 620, a second sending module 630 and a replying module 640, wherein:

[0136] The second receiving module 610 is used to receive the sound data stream transmitted by the device, and convert the sound data stream into text to obtain the question text content corresponding to the sound data stream; the sound data stream is obtained by the device performing audio coding processing on the current sound data information;

[0137] The generation module 620 is used to, upon receiving the sound collection completion message transmitted by the device, identify the user question semantics corresponding to each of the collected question text contents through a semantic recognition network, and generate a reply text content corresponding to the user question semantics through an artificial intelligence answering unit;

[0138] The second sending module 630 is used to perform a sequence audio conversion process on each sub-reply text content in the reply text content through an audio decoder according to the text sequence of the reply text content to obtain each reply audio data, and after each sub-reply text content completes the audio conversion process, respectively send the reply information containing each reply audio data to the device through a network sending program;

[0139] The reply module is used to send a preset reply completion message to the device after completing the sending process of the last reply message.

[0140] Optionally, the generating module 620 is specifically configured to:

[0141] Arrange the question text contents according to the conversion time point order of the question text contents to obtain comprehensive question text contents;

[0142] The comprehensive question text content is subjected to semantic recognition processing through a semantic recognition network to obtain semantic information corresponding to the comprehensive question text content, and question semantic information in the semantic information is extracted as user question semantics.

[0143] Optionally, the second sending module 630 is specifically configured to:

[0144] Identify a text sequence in the reply text content, and split the text sequence according to preset text reply units to obtain a target text sequence of each sub-reply text content;

[0145] According to the order of the target text sequence, each sub-reply text content is respectively processed by an audio decoder to perform sequence audio conversion processing to obtain reply audio data corresponding to each sub-reply text content.

[0146] Each module in the above-mentioned efficient transmission device for audio data can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0147] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an efficient transmission method of audio data is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0148] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps corresponding to the method for efficient transmission of audio data when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps corresponding to the method for efficient transmission of audio data are implemented.

[0151] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps corresponding to the method for efficient transmission of audio data.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An efficient transmission method for audio data, characterized in that: The method comprises: The current sound data information of the user's question is collected through the sound collection unit, and the current sound data information is subjected to audio coding processing to obtain a sound data stream; The sound data stream is sent to the server through a network sending program, and the step of collecting the sound data information of the current sound input is returned to execute, until the sound data information contains a sound data collection completion mark, and then a sound collection completion message is sent to the server; Receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information; The reply sound data is processed by sound broadcasting through a sound broadcasting unit, and the step of receiving the reply information fed back by the server is returned to be executed, until the reply completion message fed back by the server is received, and then the question reply processing is stopped.

2. The method according to claim 1, characterized in that The step of performing audio coding processing on the current sound data information to obtain a sound data stream includes: Extracting audio data and audio sequence from the current sound data information; The audio data is sequentially encoded through an audio encoder according to the audio sequence to generate a sound data stream corresponding to each time point.

3. The method according to claim 2, characterized in that The sending program through the network to send the sound data stream to the server includes: When a sound data stream at each time point is generated, the sound data stream is directly sent to the server through a network sending program.

4. The method according to claim 1, characterized in that: The voice broadcasting unit performs voice broadcasting processing on the reply voice data, including: When each reply sound data corresponding to a reply message is obtained through decoding, the sound broadcasting unit directly performs sound broadcasting processing.

5. An efficient transmission method for audio data, characterized in that: The method comprises: The sound data stream transmitted by the receiving device is converted into text by the receiving device to obtain the question text content corresponding to the sound data stream; the sound data stream is obtained by the device performing audio coding processing on the current sound data information; Upon receiving the sound collection completion message transmitted by the device, the semantic recognition network is used to identify the user question semantics corresponding to each question text content collected, and the artificial intelligence answering unit is used to generate the reply text content corresponding to the user question semantics; According to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to sequence audio conversion processing through an audio decoder to obtain each reply audio data, and after each sub-reply text content completes the audio conversion processing, the reply information containing each reply audio data is respectively sent to the device through a network sending program; After completing the sending process of the last reply message, a preset reply completion message is sent to the device.

6. The method according to claim 1, characterized in that The collected question text contents are identified through a semantic recognition network to identify the user question semantics corresponding to each question text content, including: Arrange the content of each question text according to the order of the conversion time points of each question text content to obtain comprehensive question text content; The comprehensive question text content is subjected to semantic recognition processing through a semantic recognition network to obtain semantic information corresponding to the comprehensive question text content, and question semantic information in the semantic information is extracted as user question semantics.

7. The method according to claim 1, characterized in that According to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to sequence audio conversion processing through an audio decoder to obtain each reply audio data, including: Identify a text sequence in the reply text content, and split the text sequence according to preset text reply units to obtain a target text sequence of each sub-reply text content; According to the order of the target text sequence, each sub-reply text content is respectively processed by an audio decoder to perform sequence audio conversion processing to obtain reply audio data corresponding to each sub-reply text content.

8. An efficient transmission system for audio data, characterized in that: The system includes a device and a server: The device includes a sound collection unit and a sound broadcasting unit, and the device is remotely connected to the server; The device is used to collect the current sound data information of the user's question through the sound collection unit, and perform audio coding processing on the current sound data information to obtain a sound data stream; The sound data stream is sent to the server through a network sending program, and the step of collecting the sound data information of the current sound input is returned to be executed, until the sound data information contains a sound data collection completion mark, and then a sound collection completion message is sent to the server; The server is used to receive the sound data stream transmitted by the device, and perform text conversion processing on the sound data stream to obtain the question text content corresponding to the sound data stream; when receiving the sound collection completion message transmitted by the device, for each collected question text content, the user question semantics corresponding to each question text content are identified through a semantic recognition network, and a reply text content corresponding to the user question semantics is generated through an artificial intelligence answering unit; according to the text sequence of the reply text content, each sub-reply text content in the reply text content is subjected to sequence audio conversion processing through an audio decoder to obtain each reply audio data, and after each sub-reply text content completes the audio conversion processing, the reply information containing each reply audio data is respectively sent to the device through a network sending program; after completing the sending process of the last reply message, a preset reply completion message is sent to the device; The device is used to receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information; The reply sound data is processed by sound broadcasting through a sound broadcasting unit, and the step of receiving the reply information fed back by the server is returned to be executed, until the reply completion message fed back by the server is received, and then the question reply processing is stopped.

9. An efficient transmission device for audio data, characterized in that: The device comprises: A collection module, used to collect the current sound data information of the user's question through a sound collection unit, and perform audio coding processing on the current sound data information to obtain a sound data stream; A first sending module, configured to send the sound data stream to the server through a network sending program, and return to execute the step of collecting the sound data information of the current sound input, until the sound data information contains a sound data collection completion mark, and then send a sound collection completion message to the server; The first receiving module is used to receive the reply information fed back by the server, and perform audio decoding processing on the reply information to obtain reply sound data corresponding to the reply information; The broadcast module is used to perform sound broadcast processing on the reply sound data through the sound broadcast unit, and return to execute the step of receiving the reply information feedback from the server until the reply completion message feedback from the server is received, and then stop the question reply processing.

10. An efficient transmission device for audio data, characterized in that: The device comprises: The second receiving module is used to receive the sound data stream transmitted by the device, and convert the sound data stream into text to obtain the question text content corresponding to the sound data stream; the sound data stream is obtained by the device performing audio coding processing on the current sound data information; A generation module is used to, upon receiving a sound collection completion message transmitted by the device, identify the user question semantics corresponding to each of the collected question text contents through a semantic recognition network, and generate a reply text content corresponding to the user question semantics through an artificial intelligence answering unit; A second sending module is used to perform a sequence audio conversion process on each sub-reply text content in the reply text content through an audio decoder according to the text sequence of the reply text content to obtain each reply audio data, and after each sub-reply text content completes the audio conversion process, respectively send the reply information containing each reply audio data to the device through a network sending program; The reply module is used to send a preset reply completion message to the device after completing the sending process of the last reply message.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 or 5 to 7 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 or 5 to 7 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 or 5 to 7 are implemented.

Images