Voice broadcasting method and device and voice processing and broadcasting system

Abstract

The invention discloses a voice broadcasting method, a voice broadcasting device and a voice processing and broadcasting system. The method comprises the steps that based on an input end, initial data input based on a target account are acquired, and the initial data comprise text data, audio data and user requirements; based on user requirements, the initial data are processed, target audio data are obtained, and processing operation comprises character-to-voice conversion, tone conversion and audio mixing; based on user requirements, a broadcasting party used for broadcasting the target audio data is determined, and the broadcasting party comprises at least one of an input end and a broadcasting end; and broadcasting the target audio data based on the broadcasting party. According to the invention, the technical problems of poor voice broadcast real-time performance and high cost caused by separation of the input device and the broadcast device are solved.

Description

Technical Field

[0001] The present invention relates to the field of voice broadcast technology, and in particular to a voice broadcast method, device and voice processing and broadcast system. Background Art

[0002] With the continuous advancement of Internet technology, voice communication and processing technology plays an increasingly important role in daily life, work and special demand scenarios. However, current voice conversion and broadcasting technologies face many challenges and limitations, especially in terms of device flexibility and interactive real-time performance.

[0003] Specifically, most speech conversion systems use a design that separates the input device from the broadcast device. For example, the input device may be a personal computer or a smartphone, while the broadcast device may be a dedicated speaker or a smart speaker. This separation not only increases the cost of the device, but also requires users to carry and manage multiple devices in different scenarios, which is inconvenient and redundant.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a voice broadcast method, device and voice processing and broadcast system to at least solve the technical problems of poor real-time performance and high cost of voice broadcast caused by the separation of input device and broadcast device.

[0006] According to one aspect of an embodiment of the present invention, a voice broadcast method is provided, comprising: based on an input end, obtaining initial data based on a target account input, wherein the initial data includes text data or audio data and user requirements; based on the user requirements, processing the initial data to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; based on the user requirements, determining a broadcasting party for broadcasting the target audio data, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end; based on the broadcasting party, broadcasting the target audio data.

[0007] Optionally, based on the input end, initial data based on the target account input is obtained, wherein the initial data includes text data or audio data and user requirements, including: based on the input end, receiving data to be processed based on the target account input and user requirements, wherein the data to be processed is text data or audio data; configuring the user requirements in the data to be processed to obtain data to be transmitted; when the size of the data to be transmitted exceeds a first preset threshold, compressing or slicing the data to be transmitted to obtain the initial data.

[0008] Optionally, based on user requirements, the initial data is processed to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing, including: when the initial data includes text data and user requirements, converting the text data into initial audio data; based on user requirements, determining whether to continue processing the initial audio data to obtain a determination result, wherein the determination result includes continuing processing and not processing; when the determination result is to continue processing, processing the initial audio data to obtain audio data to be transmitted, wherein the processing operations include timbre conversion and audio mixing; when the size of the audio data to be transmitted exceeds a second preset threshold, compressing or slicing the audio data to be transmitted to obtain the target audio data.

[0009] Optionally, based on a preset time interval, the connection status of a communication link is detected, wherein the communication link is used to transmit initial data and target audio data; when the connection status is interrupted, a warning notification is sent to the input end and the broadcast end.

[0010] According to another aspect of an embodiment of the present invention, a voice processing and broadcasting system is also provided, which applies any one of the above-mentioned voice broadcasting methods, including: an input end, used to obtain initial data and broadcast target audio data; a server end, used to process the initial data, obtain target audio data and determine the broadcasting party; and a broadcasting end, used to broadcast the target audio data.

[0011] Optionally, the above system also includes: a detection module, used to detect the connection status of a communication link, wherein the communication link is used to transmit initial data and target audio data; an alarm module, used to send a warning notification to the input end and the broadcast end when the connection status is interrupted.

[0012] According to another aspect of an embodiment of the present invention, a voice broadcasting device is also provided, including: an acquisition module, used to acquire initial data based on an input end based on a target account input, wherein the initial data includes text data or audio data and user requirements; a processing module, used to process the initial data based on user requirements to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; a determination module, used to determine a broadcasting party for broadcasting the target audio data based on user requirements, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end; a broadcasting module, used to broadcast the target audio data based on the broadcasting party.

[0013] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, the non-volatile storage medium including a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute any one of the above-mentioned voice broadcast methods.

[0014] According to another aspect of the embodiments of the present invention, a computer device is provided. The computer device includes a processor, and the processor is used to run a program. When the program is run, any one of the above-mentioned voice broadcast methods is executed.

[0015] According to another aspect of the embodiments of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, any one of the above-mentioned voice broadcast methods is implemented.

[0016] In an embodiment of the present invention, a voice broadcast method is adopted, and initial data based on the input of a target account is obtained based on an input end, wherein the initial data includes text data or audio data and user requirements; based on the user requirements, the initial data is processed to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; based on the user requirements, a broadcasting party for broadcasting the target audio data is determined, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end; based on the broadcasting party, the target audio data is broadcast, and the purpose of real-time broadcasting at the input end is achieved, thereby realizing the technical effect of improving the efficiency of voice broadcasting and reducing costs, and further solving the technical problem of poor real-time performance and high cost of voice broadcasting caused by the separation of the input device and the broadcasting device. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a computer terminal for implementing a voice broadcasting method is shown;

[0019] Figure 2 is a flow chart of a voice broadcast method provided according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an input terminal, i.e., a broadcasting party, provided according to an optional embodiment of the present invention;

[0021] Figure 4 It is a schematic diagram of a flow chart of inputting text data for broadcasting according to an optional embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of a process of establishing communication between three terminals according to an optional embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of a room management mechanism provided according to an optional embodiment of the present invention;

[0024] Figure 7It is a structural block diagram of a voice broadcasting device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to an embodiment of the present invention, a voice broadcast method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a voice broadcast method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the voice broadcast method in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the voice broadcast method of the above-mentioned application is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0032] Figure 2 is a flow chart of a voice broadcast method according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, based on the input end, obtaining initial data based on the target account input, wherein the initial data includes text data or audio data and user requirements.

[0034] In this step, the user can log in to the target account through the input terminal before using the speech conversion and broadcasting system. This process ensures that each user's operation is bound to their account, which facilitates the storage, call and management of personalized settings. The input terminal can collect the initial data provided by the user, which can be text input or audio input. The text input can come from the user's typing or reading from a file, while the audio data can be recorded directly through the built-in or external microphone. The collected initial data will become the basis for the subsequent speech conversion process.

[0035] At the same time, users can specify a series of requirements at the input end, which define the specific parameters of voice conversion and broadcasting, and may include audio configuration or audio configuration options. For example, users can choose different timbres, such as the timbres of men, women, children or specific virtual characters, to protect privacy or meet personalized needs; users can request to add background sound effects, volume control, playback speed adjustment, etc. to adapt to different scenarios and preferences, such as increasing the volume in a noisy environment, or adding alarm sound effects during emergency alarms; users can specify the language of voice conversion to meet the needs of multilingual communication; users can choose the broadcast end to decide whether the data is played directly at the input end or distributed to a specific independent broadcast end device through a server. The voice broadcast method provided in this embodiment can help people with language barriers to communicate in voice and improve communication skills. When the user is inconvenient to speak, the text or voice is input and converted into voice broadcast through the input end to meet daily communication needs. It can also communicate with others without leaking the real voice, protecting personal privacy.

[0036] Step S202, based on user requirements, the initial data is processed to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing.

[0037] In this step, the processing of the initial data can be performed on the server side. After the server receives the initial data from the input end, it parses the data content and performs corresponding processing according to the data type and user requirements. If the server receives text data, it can use a TTS (Text-to-Speech) engine for conversion, such as Google Cloud Text-to-Speech or Microsoft Azure TTS, to generate a voice data stream. Then, the server can use fluent-ffmpeg (open source multimedia processing framework) or other audio processing engines to perform corresponding conversion processing on the initial data or text-to-speech data passed in from the input end, such as timbre conversion and audio mixing.

[0038] Specifically, upon receiving a user's request to change the timbre, the server can use an audio processing engine to convert the timbre of the voice data. The timbre conversion technology is based on voice signal processing and machine learning, and can convert the original voice into the timbre specified by the user. The conversion operation may involve spectrum transformation, resonance peak adjustment, filtering in the time domain and frequency domain, etc., to change the sound quality and pitch of the voice, thereby achieving the conversion of the timbre. For example, the user may request that the voice be converted to the timbre of a child, the timbre of an elderly person, or the timbre of a specific virtual character to adapt to different communication scenarios or for privacy protection considerations. When the user requests audio mixing, the server can mix sound effects or music with the converted voice data or initial data to enhance the expressiveness of the voice or adapt to specific application scenarios.

[0039] After processing, the server generates an audio data packet, which is a binary data stream or a Base64-encoded audio data packet. If the audio data packet is too large, you can compress the audio, or use slice transmission to split the large file into small pieces, and send each piece of data separately to finally get the target audio data. Both of the above methods can effectively reduce data transmission time and improve transmission efficiency.

[0040] Step S203: determining a broadcasting party for broadcasting the target audio data based on the user's requirement, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end.

[0041] In this step, after the server-side processes the initial data and obtains the target audio data, it can decide to send the processing results to the broadcaster for broadcasting according to the user's requirements, wherein the broadcaster can be an input end or an independent broadcaster. If the user requires direct playback at the input end, the server can directly return the processed audio data to the input end for playback; if the user requires the use of an independent broadcasting end device, the server can broadcast the audio data to the specified broadcasting end device through the room management mechanism of Socket.IO (socket I / O). There can be many types of broadcasting end devices, including but not limited to smart speakers, car audio systems, public broadcasting systems, etc., to meet the needs of different scenarios. When the broadcaster is an independent broadcasting end, the user or system administrator can pre-configure these devices, including the unique identification of the device, network connection information, and whether it is in an activated state, etc., to ensure that the audio data can be accurately transmitted to the target broadcasting device.

[0042] In the embodiment of the present invention, the input end not only assumes the data input function, but also can be directly used as an audio playback device. Figure 3 is a schematic diagram of an input terminal, i.e., a broadcasting party, provided according to an optional embodiment of the present invention. Figure 3As shown in the figure, after inputting data, the user can hear the converted voice immediately at the input end without waiting for the data to be transmitted to other devices. This provides great convenience for scenarios that require immediate feedback, such as listening training, language learning, instant communication, etc. At the same time, independent broadcasting end devices allow the functions of the system to be physically separated to meet the needs of specific scenarios. For example, in large-scale conferences, public broadcasting or remote monitoring systems, audio data can be transmitted to multiple pre-configured broadcasting end devices through the server to achieve synchronous broadcasting or multi-point playback of audio, which improves the flexibility and application scope of the system.

[0043] Step S204: based on the announcer, announce the target audio data.

[0044] In this step, after the broadcaster is determined, the message listener set in the broadcast receives the target audio data sent by the server, performs further voice conversion or adjustment, such as audio decoding, format conversion, sound effect adjustment, etc., and then the corresponding input end or broadcast end performs voice broadcast to complete this broadcast task. For example, if the target audio data is compressed or sliced ​​before transmission, the broadcaster needs to decompress or reorganize the data after receiving the data. For sliced ​​transmitted data, the broadcaster will reassemble multiple small data blocks into a complete audio file according to the sequence information of the data packet to ensure the continuity and integrity of the audio playback.

[0045] Through the above steps, the purpose of real-time broadcasting at the input end is achieved, thereby realizing the technical effect of improving the efficiency of voice broadcasting and reducing costs, and further solving the technical problems of poor real-time performance and high cost of voice broadcasting caused by the separation of input devices and broadcasting devices.

[0046] As an optional embodiment, based on the input end, initial data based on the target account input is obtained, wherein the initial data includes text data or audio data and user requirements, including: based on the input end, receiving data to be processed based on the target account input and user requirements, wherein the data to be processed is text data or audio data; configuring the user requirements in the data to be processed to obtain data to be transmitted; when the size of the data to be transmitted exceeds a first preset threshold, compressing or slicing the data to be transmitted to obtain the initial data.

[0047] Optionally, the input end serves as the first interface for the user to interact with the system, and is responsible for receiving the data to be processed input by the user based on the target account, which can be text input (such as typing input) or direct audio data (such as voice recorded through a microphone). At the same time, the user can set personalized conversion and playback requirements at the input end, such as timbre, sound effects, playback speed, etc. These user requirements will be encapsulated together with the data to be processed to form the data to be transmitted. Before data transmission, the size of the data to be transmitted can be checked. If the data size exceeds the preset first threshold, in order to reduce the transmission time and improve the transmission efficiency, compression or slicing technology can be used to reduce the data volume. Compression technology usually uses a general compression algorithm, such as zlib, which can reduce the data volume while maintaining data quality. Slicing is to divide a large audio file into multiple small data packets and send them in batches to reduce the amount of data transmitted in a single time and improve the stability of data transmission.

[0048] As an optional embodiment, based on user requirements, the initial data is processed to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing, including: when the initial data includes text data and user requirements, converting the text data into initial audio data; based on user requirements, judging whether to continue processing the initial audio data to obtain a judgment result, wherein the judgment result includes continuing processing and not processing; when the judgment result is to continue processing, processing the initial audio data to obtain audio data to be transmitted, wherein the processing operations include timbre conversion and audio mixing; when the size of the audio data to be transmitted exceeds a second preset threshold, compressing or slicing the audio data to be transmitted to obtain the target audio data.

[0049] Optionally, Figure 4 is a flow chart of inputting text data for broadcasting according to an optional embodiment of the present invention. Figure 4As shown, when the input end receives text data, the text data can be converted into initial audio data using TTS technology. This step converts the text information input by the user into a voice signal that can be broadcast, laying the foundation for subsequent audio processing and playback. Based on the user's requirements (such as changing the timbre, adding sound effects, etc.) proposed by the user through the input end, it is determined whether the initial audio data generated by TTS needs to be further processed. If the user's requirements include operations such as timbre conversion or audio mixing, the subsequent audio processing steps will be executed; if the user only requires basic TTS conversion, this step will be skipped directly to reduce unnecessary processing time. In the case of determining that further processing is required, the audio processing module on the server side will continue to process the initial audio data, for example, converting the voice into the timbre of children, the timbre of the elderly, or the timbre of a specific virtual character to meet the personalized needs of the user, and can also combine sound effects or background music with audio data to enhance the expressiveness and adaptability of the audio. After the audio data processing is completed, the size of the audio data to be transmitted can be checked. If the data size exceeds the preset second threshold, compression technology (such as zlib compression) or slice transmission strategy will be used to reduce transmission time and improve network transmission efficiency. Compression technology can reduce the data volume while maintaining audio quality, while slice transmission can improve the stability of transmission and ensure the complete transmission of large files. Finally, the broadcaster is selected based on user requirements to broadcast the target audio data.

[0050] As an optional embodiment, the connection status of a communication link is detected based on a preset time interval, wherein the communication link is used to transmit initial data and target audio data; when the connection status is interrupted, a warning notification is sent to the input end and the broadcast end.

[0051] Optionally, in this embodiment, the input end, the broadcast end and the server end may establish a WebSocket (network socket) connection through Socket.IO. Figure 5 is a schematic diagram of a process of establishing communication between three terminals according to an optional embodiment of the present invention. Figure 5As shown, the PC end is the input end, and the client end is the broadcast end. The three ends can perform handshake and verification to confirm that the connection state is normal and process the initial configuration after the connection is established. Specifically, the input end and the broadcast end can first initiate a WebSocket connection request through Socket.IO, the server end accepts the connection request and performs handshake verification to confirm the connection state, and finally the server end returns a connection confirmation message. After the input end and the broadcast end receive the confirmation message, they enter the connection state. In order to ensure the reliability of the above connection state, the connection state of the communication link can be detected regularly. A heartbeat detection mechanism can be used, that is, a heartbeat signal is sent to detect the connection state of the communication link within a preset time interval (such as every 5 seconds or 10 seconds). The heartbeat signal can be a lightweight data packet used to test whether the network connection is normal and calculate the network delay. If the heartbeat detection mechanism finds that the connection state of the communication link is interrupted, an early warning notification can be immediately sent to the input end and the broadcast end. The early warning notification is used to remind users that there may be problems with the communication link and corresponding measures need to be taken, such as re-establishing the connection, checking the network environment or device status, etc., to restore the normal operation of the service.

[0052] Through regular heartbeat detection, connection interruptions can be discovered in time, avoiding long periods of ineffective waiting, and improving the real-time and reliability of services. Early warning notifications enable users to quickly realize communication problems and take timely measures, reducing voice service interruptions caused by network interruptions and improving the overall user experience. If the connection status is not checked regularly, communication may be interrupted for a long time due to network problems without the user noticing, resulting in unstable voice conversion and broadcast services and poor user experience.

[0053] According to an embodiment of the present invention, a voice processing and broadcasting system applying the above-mentioned voice broadcasting method is also provided, including: an input end, used to obtain initial data and broadcast target audio data; a server end, used to process the initial data, obtain target audio data and determine the broadcasting party; and a broadcasting end, used to broadcast the target audio data.

[0054] Optionally, the speech processing and broadcasting system may include an input end, a server end and a broadcasting end.

[0055] The input end is the first interface for the system to interact with the user. The user enters the text data to be broadcast or records the voice data at the input end. The user chooses to change the timbre, special sound effect processing and other options according to the needs to obtain the user's requirements. Then the input end can encapsulate the text or voice data and the user's requirements in the request body to obtain the initial data. Finally, the input end sends the initial data to the server. The input end can also compress or slice the text or voice data that is too large. In addition, the input end can also be used as a broadcaster in some scenarios to achieve the integration of input and broadcast. A listener can be created at the input end to receive the target audio data from the server.

[0056] The server is the central processing unit of the entire system, and undertakes the core functions of data processing and scheduling. Before the broadcast task begins, the server can create a room according to user requirements and create a separate room for each broadcast task. Figure 6 is a schematic diagram of a room management mechanism provided according to an optional embodiment of the present invention, such as Figure 6 As shown, when the input end and the broadcast end are connected to the server end, the server end allocates them to their respective corresponding rooms, and uses the room function of Socket.IO to broadcast the data of one input end to multiple broadcast parties for broadcasting. When the server end receives the initial data sent by the input end based on the message listener, if it is compressed data, it needs to be decompressed, if it is sliced, it needs to be spliced ​​in sequence, and then it is determined whether it is text data or audio data. If it is text data, the TTS engine is called to convert the text into initial audio data. Further, according to user requirements, it is determined whether the initial audio data after text conversion is directly broadcast or needs to be specially processed again. If the converted audio does not need to be specially processed, pre-processing is performed before sending it to the broadcast party. According to user requirements, the recorded voice data or the audio data after text-to-speech conversion is converted accordingly, such as changing the timbre, special sound effect processing, etc. After the processing is completed, the server end encapsulates the generated audio binary data stream or Base64-encoded data into a data packet. If the audio data packet is too large, the compression tool is used to compress the audio, or the large file is split into small pieces using slice transmission to obtain the target audio data. Before sending the target audio data, the server first determines the broadcaster, whether it is the input end broadcasting or the broadcast end broadcasting. According to the user's requirements, the target audio data is sent to the corresponding room and broadcast processing is performed on the broadcaster in the room.

[0057] The broadcast end is responsible for receiving the audio data sent by the server, and performing operations such as decoding, reorganization (if the audio data is transmitted in slices), and pre-playback configuration, and then playing the audio data to complete the voice broadcast task. Specifically, the broadcast end starts the broadcast processing. You can use AudioContext (audio environment) to create an audio node, connect the audio source node to the target node of AudioContext (device speaker), and then call the broadcast method of the audio source node to play the audio, and the broadcast task ends. The independent broadcast end improves the flexibility of the system and the breadth of application scenarios. It can be deployed in various environments, such as homes, offices, public places, etc. to meet different needs.

[0058] As an optional embodiment, the above system also includes: a detection module, used to detect the connection status of the communication link, wherein the communication link is used to transmit initial data and target audio data; an alarm module, used to send a warning notification to the input end and the broadcast end when the connection status is interrupted.

[0059] Optionally, the core function of the detection module is to monitor the health of the communication link, that is, the connection status between the input end and the server end, and the server end and the broadcast end. It evaluates the stability of the connection by regularly sending and receiving heartbeat signals. After receiving the heartbeat signal, the detection module will analyze the response time and content of the signal to determine whether the communication link is normal. If the signal response times out or the content is abnormal, it may indicate that the connection status is unstable or interrupted. The alarm module is responsible for sending an early warning notification to the input end and the broadcast end when the detection module identifies that the communication link connection status is interrupted. When the detection module determines that the communication link is interrupted, the alarm module will generate an early warning notification, which contains detailed information about the interruption, such as the time of the interruption, possible causes, etc. After the input end and the broadcast end receive the early warning notification, the user will be informed of the current connection problem.

[0060] The speech processing and broadcasting system provided by the embodiment of the present invention can help people with speech impairments and ordinary users to communicate by voice, and protect user privacy through text-to-speech conversion and timbre conversion. The system integrates text and speech processing, supports user recording and a variety of sound effect processing, provides personalized voice output, and satisfies users' personalized settings for voice timbre, volume, playback speed, etc. WebSocket communication ensures real-time performance and low latency, making the system suitable for scenarios that require rapid response, such as instant communication and emergency alerts. When a single device is broadcasting, it can achieve input and broadcast, reducing equipment requirements and costs. At the same time, it can support the server to create an independent room for each broadcast end session to realize message broadcasting.

[0061] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that the voice broadcast method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, a disk, an optical disk), including a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0063] According to an embodiment of the present invention, a device for implementing the above-mentioned voice broadcast method is also provided. Figure 7 is a structural block diagram of a voice broadcasting device provided according to an embodiment of the present invention, such as Figure 7 As shown, the voice broadcasting device includes: an acquisition module 71, a processing module 72, a determination module 73 and a broadcasting module 74. The device is described below.

[0064] The acquisition module 71 is used to acquire initial data based on the target account input based on the input end, wherein the initial data includes text data or audio data and user requirements.

[0065] The processing module 72 is connected to the acquisition module 71 and is used to process the initial data based on user requirements to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing.

[0066] The determination module 73 is connected to the processing module 72 and is used to determine a broadcasting party for broadcasting the target audio data based on user requirements, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end.

[0067] The broadcasting module 74 is connected to the determination module 73 and is used to broadcast the target audio data based on the broadcasting party.

[0068] It should be noted that the acquisition module 71, the processing module 72, the determination module 73 and the broadcast module 74 correspond to steps S201 to S204 in the embodiment, and the examples and application scenarios implemented by the multiple modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in the embodiment.

[0069] An embodiment of the present invention may provide a computer device. Optionally, in this embodiment, the computer device may be located in at least one network device among multiple network devices of a computer network. The computer device includes a memory and a processor.

[0070] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the voice broadcast method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned voice broadcast method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0071] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: based on the input end, obtain initial data based on the target account input, wherein the initial data includes text data or audio data and user requirements; based on the user requirements, process the initial data to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; based on the user requirements, determine the broadcaster for broadcasting the target audio data, wherein the broadcaster includes at least one of the following: an input end and a broadcaster end; based on the broadcaster, broadcast the target audio data.

[0072] According to an embodiment of the present invention, a voice broadcast method is provided. Based on an input end, initial data based on a target account input is obtained, wherein the initial data includes text data or audio data and user requirements; based on user requirements, the initial data is processed to obtain target audio data, wherein the processing operation includes text-to-speech, timbre conversion and audio mixing; based on user requirements, a broadcasting party for broadcasting the target audio data is determined, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end; based on the broadcasting party, the target audio data is broadcast, and the purpose of real-time broadcasting at the input end is achieved, thereby achieving the technical effect of improving the efficiency of voice broadcasting and reducing costs, thereby solving the technical problem of poor real-time performance and high cost of voice broadcasting caused by the separation of the input device and the broadcasting device.

[0073] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a non-volatile storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0074] The embodiment of the present invention further provides a non-volatile storage medium. Optionally, in this embodiment, the non-volatile storage medium can be used to store the program code executed by the voice broadcast method provided in the above embodiment.

[0075] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0076] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: based on the input end, obtaining initial data based on the target account input, wherein the initial data includes text data or audio data and user requirements; based on the user requirements, processing the initial data to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; based on the user requirements, determining a broadcaster for broadcasting the target audio data, wherein the broadcaster includes at least one of the following: an input end and a broadcaster end; based on the broadcaster, broadcasting the target audio data.

[0077] An embodiment of the present invention also provides a computer program product, including a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it can achieve: based on an input end, obtaining initial data based on a target account input, wherein the initial data includes text data or audio data and user requirements; based on user requirements, processing the initial data to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; based on user requirements, determining a broadcaster for broadcasting the target audio data, wherein the broadcaster includes at least one of the following: an input end and a broadcaster end; based on the broadcaster, broadcasting the target audio data.

[0078] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0079] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0082] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0084] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A voice broadcast method, characterized in that: include: Based on the input end, obtaining initial data based on the target account input, wherein the initial data includes text data or audio data and user requirements; Based on the user's requirement, the initial data is processed to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing; Based on the user's requirement, determining a broadcasting party for broadcasting the target audio data, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end; Based on the broadcasting party, the target audio data is broadcasted.

2. The method according to claim 1, characterized in that The step of obtaining initial data based on the target account input based on the input end, wherein the initial data includes text data or audio data and user requirements, includes: Based on the input end, receiving the data to be processed based on the target account input and the user requirement, wherein the data to be processed is the text data or the audio data; Configuring the user requirement in the data to be processed to obtain data to be transmitted; When the size of the data to be transmitted exceeds a first preset threshold, the data to be transmitted is compressed or sliced ​​to obtain the initial data.

3. The method according to claim 1, characterized in that The initial data is processed based on the user's requirement to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion and audio mixing, including: In a case where the initial data includes the text data and the user's request, converting the text data into initial audio data; Based on the user's requirement, determining whether to continue processing the initial audio data, and obtaining a determination result, wherein the determination result includes continuing processing and not processing; If the judgment result is to continue processing, the initial audio data is processed to obtain audio data to be transmitted, wherein the processing operation includes timbre conversion and audio mixing; When the size of the audio data to be transmitted exceeds a second preset threshold, the audio data to be transmitted is compressed or sliced ​​to obtain the target audio data.

4. The method according to any one of claims 1 to 3, characterized in that: Also includes: Based on a preset time interval, detecting a connection state of a communication link, wherein the communication link is used to transmit the initial data and the target audio data; When the connection state is interrupted, a warning notification is sent to the input end and the broadcast end.

5. A speech processing and broadcasting system, characterized in that: The method for voice broadcasting according to any one of claims 1 to 4 is applied, comprising: Input terminal, used to obtain initial data and broadcast target audio data; The server is used to process the initial data, obtain the target audio data and determine the broadcaster; The broadcasting end is used to broadcast the target audio data.

6. The system according to claim 5, characterized in that Also includes: a detection module, configured to detect a connection state of a communication link, wherein the communication link is used to transmit the initial data and the target audio data; The alarm module is used to send an early warning notification to the input end and the broadcast end when the connection state is interrupted.

7. A voice broadcasting device, characterized in that: include: An acquisition module, configured to acquire initial data inputted based on a target account based on an input terminal, wherein the initial data includes text data or audio data and user requirements; A processing module, configured to process the initial data based on the user's requirements to obtain target audio data, wherein the processing operations include text-to-speech, timbre conversion, and audio mixing; A determination module, used for determining a broadcasting party for broadcasting the target audio data based on the user requirement, wherein the broadcasting party includes at least one of the following: an input end and a broadcasting end; The broadcasting module is used to broadcast the target audio data based on the broadcasting party.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the voice broadcast method according to any one of claims 1 to 4.

9. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is running, the processor executes the voice broadcast method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the voice broadcast method described in any one of claims 1 to 4 is implemented.

Images