Voice signal processing method and device, terminal equipment and storage medium
By establishing wireless connections between audio devices and optimizing data transmission modes, the problems of voice processing latency and high power consumption in international communication have been solved, enabling real-time language conversion and clear recognition, thus improving user experience and device applicability.
Patent Information
- Application Number
- CN202310595538.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-11-26
AI Technical Summary
Existing voice processing equipment suffers from problems such as real-time voice processing latency, high power consumption, low recognition accuracy, and limited applicability in international communication, especially performing poorly in noisy environments.
By receiving voice data from a first audio device, generating voice data in a second language, establishing a wireless connection between the first and second audio devices, performing real-time language conversion using an intelligent translation model, optimizing data transmission modes and security verification, and reducing device power consumption.
It enables clear speech recognition in noisy environments, expands the applicable scenarios for speech processing and the duration of device conversations, reduces device power consumption, and improves user experience and security.
Smart Images

Figure CN119028331A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of audio signals, and in particular to a method and apparatus for processing voice signals, a terminal device, and a storage medium. Background Art
[0002] With the continuous development and progress of international trade and communication, the trend of user internationalization will inevitably lead to communication problems between people of different languages. Taking the conversation between Chinese and English native speakers as an example, Chinese users need to be proficient in English to communicate with English users, and few English users understand Chinese. Therefore, language has become the biggest obstacle to international communication, and the demand for real-time voice processing during calls has become increasingly important.
[0003] At present, in the related technical solutions, during the conversation between users, the voice processing device (for example, headphones) cannot perform voice processing (for example, translation) in real time, or there is a delay in voice processing. At the same time, in the long-term conversation scenario, the voice processing device consumes a lot of power and is difficult to work continuously. In addition, when the voice processing device recognizes voice in a noisy environment, it may result in low recognition accuracy and a small scope of application. Summary of the invention
[0004] The embodiments of the present disclosure provide a voice signal processing method and apparatus, a terminal device, and a storage medium.
[0005] A first aspect of the present disclosure provides a method for processing a speech signal, the method comprising:
[0006] Based on a first wireless connection with a first audio device, receiving first voice data sent by the first audio device; wherein the first voice data is voice data in a first language;
[0007] Generate second voice data according to the first voice data; wherein the second voice data is voice data in a second language;
[0008] Based on at least one second wireless connection with a second audio device, the second voice data is sent to at least one second audio device, wherein the second voice data is at least used for output by the second audio device.
[0009] Based on the above solution, the method further includes:
[0010] Acquire first information of the first audio device; wherein the first information is at least used to determine a data transmission mode supported by the first audio device;
[0011] A target data transmission mode of the first wireless connection is determined according to the first information.
[0012] Based on the above solution, the first information includes at least one of the following:
[0013] Hardware information of the first audio device; wherein the hardware information indicates the hardware capabilities of the first audio device; different data transmission modes require different hardware capabilities;
[0014] The mode information of the first audio device; the mode information indicates a transmission mode supported by the first audio device.
[0015] Based on the above solution, the method further includes:
[0016] The mode information of the target data transmission mode is sent to the first audio device, wherein the target data transmission mode is used for data communication of the first wireless connection.
[0017] Based on the above solution, the method further includes:
[0018] Acquire second information of the second audio device, wherein the second information is at least used to determine a data transmission mode supported by the second audio device;
[0019] A target data transmission mode of the second wireless connection is determined according to the second information.
[0020] Based on the above solution, the second information includes at least one of the following:
[0021] Hardware information of the second audio device; wherein the hardware information indicates the hardware capabilities of the second audio device; different data transmission modes require different hardware capabilities;
[0022] The mode information sent by the second audio device, wherein the mode information indicates a transmission mode supported by the second audio device.
[0023] Based on the above solution, the method further includes:
[0024] The mode information of the target data transmission mode is sent to the second audio device, wherein the target data transmission mode is used for data communication of the second wireless connection.
[0025] Based on the above solution, the target data transmission mode includes at least one of the following:
[0026] Ultra-wideband UWB;
[0027] Wireless Fidelity WIFI;
[0028] Bluetooth.
[0029] Based on the above solution, generating second voice data according to the first voice data includes:
[0030] The first speech data and the language identifier of the second language are input into an intelligent translation model to obtain the second speech data.
[0031] Based on the above solution, generating second voice data according to the first voice data includes:
[0032] Determining whether security verification is required for data transmission over the first wireless connection;
[0033] When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data;
[0034] The second voice data is generated according to the first voice data that has passed the security verification.
[0035] Based on the above solution, sending the second voice data to at least one second audio device based on at least one second wireless connection with the second audio device includes:
[0036] determining whether security verification is required for data transmission of the second wireless connection;
[0037] When the data transmission of the second wireless connection requires security verification, using the security unit to perform security verification on the second voice data;
[0038] The second voice data that passes the security verification is sent to the second audio device.
[0039] Based on the above solution, before receiving the first voice data sent by the first audio device, the method further includes:
[0040] When an operation acting on a first control on the call interface is detected, the first wireless connection and the second wireless connection are established respectively.
[0041] Based on the above solution, the method further includes:
[0042] When an operation acting on a second control on the call interface is detected, text information of the first voice data and the second voice data is displayed.
[0043] Based on the above solution, the method further includes:
[0044] Displaying device information of the second audio device on the language setting interface;
[0045] When an operation acting on a third control on the language setting interface is detected, obtaining operation information based on the third control;
[0046] A language identifier of the second language of the second audio device is determined according to the operation information.
[0047] A second aspect of the present disclosure provides a method for processing a speech signal, which is applied to a first audio device. The method includes:
[0048] Sending mode information based on the first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device;
[0049] Receive mode information of a target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send first voice data in the target transmission mode, an end having the first wireless connection with the first audio device receives the first voice data in the target transmission mode, generates second voice data based on the first voice data, and sends the second voice data to at least one second audio device.
[0050] Based on the above solution, the target data transmission mode includes at least one of the following:
[0051] Ultra-wideband UWB;
[0052] Wireless Fidelity WIFI;
[0053] Bluetooth.
[0054] Based on the above solution, sending the first voice data collected by the first audio device based on the first wireless connection includes:
[0055] Determining whether security verification is required for data transmission over the first wireless connection;
[0056] When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data;
[0057] The first voice data after security verification is sent.
[0058] A third aspect of the present disclosure provides a method for processing a speech signal, which is applied to a second audio device, and the method includes:
[0059] Sending mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device;
[0060] receiving mode information of a target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data;
[0061] The second voice data is generated by receiving the first voice data sent by the first audio device at one end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and generating the second voice data according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
[0062] Based on the above solution, the target data transmission mode includes at least one of the following:
[0063] Ultra-wideband UWB;
[0064] Wireless Fidelity WIFI;
[0065] Bluetooth.
[0066] Based on the above solution, the method further includes:
[0067] determining whether security verification is required for data transmission of the second wireless connection;
[0068] When the data transmission of the second wireless connection requires security verification, the security unit is used to perform security verification on the second voice data.
[0069] A fourth aspect of the present disclosure provides a speech signal processing device, the device comprising:
[0070] A first receiving module, configured to receive first voice data sent by a first audio device based on a first wireless connection with the first audio device; wherein the first voice data is voice data in a first language;
[0071] A generating module, configured to generate second voice data according to the first voice data; wherein the second voice data is voice data in a second language;
[0072] The first sending module is configured to send the second voice data to at least one second audio device based on at least one second wireless connection with the second audio device, wherein the second voice data is at least used for output of the second audio device.
[0073] Based on the above solution, the device also includes:
[0074] A first acquisition module, configured to acquire first information of the first audio device; wherein the first information is at least used to determine a data transmission mode supported by the first audio device;
[0075] A first determining module is configured to determine a target data transmission mode of the first wireless connection according to the first information.
[0076] Based on the above solution, the first information includes at least one of the following:
[0077] Hardware information of the first audio device; wherein the hardware information indicates the hardware capabilities of the first audio device; different data transmission modes require different hardware capabilities;
[0078] The mode information of the first audio device; the mode information indicates a transmission mode supported by the first audio device.
[0079] Based on the above solution, the device also includes:
[0080] The second receiving module is used to send mode information of the target data transmission mode to the first audio device, wherein the target data transmission mode is used for data communication of the first wireless connection.
[0081] Based on the above solution, the device also includes:
[0082] A second acquisition module, configured to acquire second information of the second audio device, wherein the second information is at least used to determine a data transmission mode supported by the second audio device;
[0083] A second determining module is configured to determine a target data transmission mode of the second wireless connection according to the second information.
[0084] Based on the above solution, the second information includes at least one of the following:
[0085] Hardware information of the second audio device; wherein the hardware information indicates the hardware capabilities of the second audio device; different data transmission modes require different hardware capabilities;
[0086] The mode information sent by the second audio device, wherein the mode information indicates a transmission mode supported by the second audio device.
[0087] Based on the above solution, the device also includes:
[0088] The third sending module is used to send mode information of the target data transmission mode to the second audio device, wherein the target data transmission mode is used for data communication of the second wireless connection.
[0089] Based on the above solution, the target data transmission mode includes at least one of the following:
[0090] Ultra-wideband UWB;
[0091] Wireless Fidelity WIFI;
[0092] Bluetooth.
[0093] Based on the above solution, the generation module is specifically used to input the first voice data and the language identifier of the second language into the intelligent translation model to obtain the second voice data.
[0094] Based on the above scheme, the generation module is specifically used to determine whether the data transmission of the first wireless connection requires security verification; when the data transmission of the first wireless connection requires security verification, the first voice data is security verified using the security unit; and the second voice data is generated based on the first voice data that passes the security verification.
[0095] Based on the above scheme, the first sending module is specifically used to determine whether the data transmission of the second wireless connection requires security verification; when the data transmission of the second wireless connection requires security verification, the second voice data is security verified using the security unit; and the second voice data that passes the security verification is sent to the second audio device.
[0096] Based on the above solution, the device also includes:
[0097] The connection establishment module is used to establish the first wireless connection and the second wireless connection respectively when an operation acting on the first control on the call interface is detected.
[0098] Based on the above solution, the device also includes:
[0099] The first display module is used to display text information of the first voice data and the second voice data when an operation acting on the second control on the call interface is detected.
[0100] Based on the above solution, the device also includes:
[0101] A second display module, used for displaying the device information of the second audio device on the language setting interface;
[0102] A third acquisition module, configured to acquire operation information based on the third control when an operation acting on the third control on the language setting interface is detected;
[0103] A third determining module is configured to determine a language identifier of the second language of the second audio device according to the operation information.
[0104] A fifth aspect of the embodiments of the present disclosure provides a speech signal processing device, which is applied to a first audio device, and the device includes:
[0105] a fourth sending module, configured to send mode information based on the first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device;
[0106] A second receiving module is used to receive mode information of a target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send first voice data in the target transmission mode, an end having the first wireless connection with the first audio device to receive the first voice data in the target transmission mode, generate second voice data based on the first voice data, and send the second voice data to at least one second audio device.
[0107] Based on the above solution, the target data transmission mode includes at least one of the following:
[0108] Ultra-wideband UWB;
[0109] Wireless Fidelity WIFI;
[0110] Bluetooth.
[0111] Based on the above scheme, the fourth sending module is specifically used to determine whether the data transmission of the first wireless connection requires security verification; when the data transmission of the first wireless connection requires security verification, use the security unit to perform security verification on the first voice data; and send the first voice data after security verification.
[0112] A sixth aspect of the present disclosure provides a speech signal processing device, which is applied to a second audio device. The device includes:
[0113] a fifth sending module, configured to send mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device;
[0114] A third receiving module is used to receive mode information of a target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data; the second voice data is the first voice data sent by the first audio device received by the end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and the second voice data is generated according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
[0115] Based on the above solution, the target data transmission mode includes at least one of the following:
[0116] Ultra-wideband UWB;
[0117] Wireless Fidelity WIFI;
[0118] Bluetooth.
[0119] Based on the above solution, the device also includes:
[0120] A fourth determination module, used to determine whether the data transmission of the second wireless connection requires security verification;
[0121] The security verification module is used to perform security verification on the second voice data using the security unit when the data transmission of the second wireless connection needs to be security verified.
[0122] According to a seventh aspect of an embodiment of the present disclosure, a terminal device is provided, including:
[0123] a memory for storing processor-executable instructions;
[0124] A processor connected to the memory;
[0125] Wherein, the processor is configured to execute the speech signal processing method as described above.
[0126] According to an eighth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a computer, the computer is enabled to execute the speech signal processing method as described above.
[0127] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:
[0128] The voice signal processing method in the embodiment of the present disclosure, on the one hand, receives the first voice data sent by the first audio device, and generates the second voice data according to the first voice data. Thus, the second voice data is sent to at least one second audio device, and the voice data is transmitted between multiple audio devices, so that even the user wearing the first audio device and the second audio device can clearly recognize the voice in a noisy environment. In this way, not only the real-time processing of voice data in different languages is realized, so as to facilitate the voice conversation between multiple users proficient in different languages, but also the applicable scene of voice signal processing is improved. On the other hand, by establishing a first wireless connection with the first audio device and establishing a second wireless connection with the second audio device, the second voice data is generated according to the first voice data to realize the conversion between the first language and the second language. In this way, the first audio device only needs to send the first voice data, and the second audio device receives the second voice data. The first audio device and the second audio device need to perform less processing, and the hardware requirements for the first audio device and the second audio device are low. Therefore, the voice signal processing method can be applied to most audio devices, thereby further improving the applicable scene of the voice signal, and effectively reducing the power consumption of the first audio device and the second audio device, and improving the voice conversation time. BRIEF DESCRIPTION OF THE DRAWINGS
[0129] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0130] Figure 1 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0131] Figure 2 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0132] Figure 3 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0133] Figure 4 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0134] Figure 5 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0135] Figure 6 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0136] Figure 7is a flow chart of a method for processing a speech signal according to an exemplary embodiment;
[0137] Figure 8 is a structural schematic diagram of a speech signal processing device according to an exemplary embodiment;
[0138] Fig. 9 is a structural schematic diagram of a speech signal processing device according to an exemplary embodiment;
[0139] Fig.10 is a structural schematic diagram of a speech signal processing device according to an exemplary embodiment;
[0140] Fig.11 is a structural schematic diagram of a speech signal processing device according to an exemplary embodiment;
[0141] Fig.12 It is a schematic diagram of the structure of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION
[0142] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.
[0143] like Figure 1 As shown, the present disclosure provides a method for processing a speech signal, the method comprising:
[0144] S1100: receiving first voice data sent by a first audio device based on a first wireless connection with the first audio device; wherein the first voice data is voice data in a first language;
[0145] S1200: Generate second voice data according to the first voice data; wherein the second voice data is voice data in a second language;
[0146] S1300: Send the second voice data to at least one second audio device based on at least one second wireless connection with the second audio device, wherein the second voice data is at least used for output of the second audio device.
[0147] The speech signal processing method provided in the embodiments of the present disclosure can be executed in any terminal device, and the terminal device may include various electronic devices, servers or clouds with wireless communication functions. The electronic devices herein may include mobile terminals and fixed terminals. By way of example, mobile terminals include but are not limited to smart phones, smart TVs, tablet computers, laptops, smart cars or wearable devices (e.g., smart watches, etc.). By way of example, fixed terminals include but are not limited to desktop computers, etc.
[0148] The above-mentioned first wireless connection and second wireless connection include but are not limited to any short-range wireless connection. Exemplarily, the short-range wireless connection includes but is not limited to short-range communication networks such as Wireless Fidelity (WIFI), Bluetooth, Zigbee or near field communication (NFC) network.
[0149] The first language and the second language are natural languages, including any language. For example, the first language and the second language include but are not limited to Chinese, English, French or Cantonese.
[0150] The first voice data is the sound produced by wearing the first audio device. The second voice data is the sound produced by wearing the second audio device.
[0151] The first audio device and the second audio device may be audio devices of the same or different product types. The first audio device and the second audio device are any devices with audio input and / or audio output functions. The first audio device and the second audio device are any devices with wireless communication functions. Exemplarily, the first audio device and the second audio device include but are not limited to wireless headphones, speakers or microphones, etc.
[0152] The first audio device and the second audio device may be the same audio devices or different audio devices. Exemplarily, the first audio device and the second audio device may be two wireless headphones. Exemplarily, the first audio device is a wireless headset and the second audio device is a speaker. Exemplarily, the first audio device may be any one of a pair of wireless headphones and the second audio device may be the other of the pair of wireless headphones.
[0153] The voice signal processing method provided by the embodiment of the present disclosure can be applied to the conversation between multiple users. For example, user A wears a first audio device, user B wears a second audio device, user A sends voice data in Chinese as the first language, and user B sends language data in English as the second language. User B is not proficient in Chinese, and can turn on the call function on the call interface in the terminal device to turn on the voice translation function. Specifically, through the call, the first voice data sent by user A is translated into the second voice data that user B is proficient in, thereby facilitating the communication between user A and user B.
[0154] The voice signal processing method provided in the embodiments of the present disclosure can be applied, for example, in scenarios where a second voice is used for output in meetings, chatting, socializing, or when making short videos.
[0155] In one embodiment, the first voice data is voice data collected by the first audio device. When it is detected that the first voice data is a dialect, that is, a local language, the first voice data can be converted into a standardized language, and then the second voice data is generated based on the first voice data. In this way, the voice signal processing is more intelligent and the user experience is improved.
[0156] In one embodiment, the language identifier of the first voice data can be automatically identified, that is, there is no need to set the language identifier of the first language corresponding to the first voice data, and the language identifier of the first language is automatically identified through an automatic recognition program, so that the first voice data is recognized according to the first language, and the second voice data with the language identifier of the second language is generated according to the recognized first voice data. In this way, by automatically identifying the language identifier of the first language corresponding to the first voice data, the voice signal processing is made more intelligent, and the user experience is improved.
[0157] The first audio device and the second audio device may be devices with small battery capacity. Exemplarily, the battery capacity of the first audio device and the second audio device may be smaller than the battery capacity of the terminal device.
[0158] Since the first audio device and the second audio device are usually portable devices, and the battery capacity of the portable device is small, if the program in step S1200 is placed in the first audio device or the second audio device, it may cause the power consumption of the first audio device or the second audio device to be high, and it is impossible to perform conversation translation for a long time, which reduces the user experience. The embodiment of the present disclosure is implemented by placing some processing programs, such as the program in step S1200, in the terminal device. The terminal device usually has sufficient power and relatively good performance. By utilizing the advantages of the terminal device, the process of users talking through the first audio device and the second audio device is more stable, and the power consumption of the first audio device and the second audio device can be reduced, which improves the user's experience of making calls.
[0159] In step S1300, the terminal device may establish a second wireless connection with one or more second audio devices, and different second audio devices may correspond to different second languages. For example, taking two second audio devices as an example, user C and user D each wear a second audio device, user C wears second audio device 1, and user D wears second audio device 2. The second language corresponding to second audio device 1 may be English, and the second language corresponding to second audio device 2 may be French.
[0160] In one embodiment, the language identifiers of the second languages corresponding to the multiple second audio devices may be the same or different. Therefore, before S1200, the method further includes: determining the language identifier of the second language corresponding to each second audio device; in S1200, the second voice data corresponding to each second audio device can be generated according to the first voice data. Thus, the second voice data corresponding to the second audio device can be sent to the corresponding second audio device, respectively, so as to facilitate the second audio device to output the corresponding second voice data. In this way, it is convenient to communicate with people who are proficient in different languages, and the user's conversation experience is improved.
[0161] Since the first audio device and the second audio device may send or receive audio signals at the same time, in order to reduce the signal interference between the first audio device and the second audio device, the first wireless connection and the second wireless connection may use frequencies of different frequency bands to send and receive voice data. For example, the frequency can be divided into a first frequency band, a second frequency band, and a third frequency band, wherein the frequency of the first frequency band is lower than the frequency of the second frequency band, and the frequency of the second frequency band is lower than the frequency of the third frequency band. Exemplarily, the first wireless connection may use the frequency of the first frequency band to receive the first voice data, and the second wireless connection may use the frequency of the second frequency band to send the second voice data. In this way, the possibility of signal interference between the first audio device and the second audio device can be reduced.
[0162] In S1300, the second voice data can be output to the second audio device, and the output method includes but is not limited to voice output or text output. The output method can be set in the terminal device or the second audio device. By supporting multiple output methods, it is convenient for users to use in multiple scenarios, thereby improving the user experience.
[0163] The voice signal processing method in the embodiment of the present disclosure, on the one hand, receives the first voice data sent by the first audio device, and generates the second voice data according to the first voice data. Thus, the second voice data is sent to at least one second audio device, and the voice data is transmitted between multiple audio devices, so that even the user wearing the first audio device and the second audio device can clearly recognize the voice in a noisy environment. In this way, not only the real-time processing of voice data in different languages is realized, so as to facilitate the voice conversation between users proficient in different languages, but also the applicable scenarios of voice signal processing are improved. On the other hand, by establishing a first wireless connection with the first audio device and a second wireless connection with the second audio device, the second voice data is generated according to the first voice data to realize the conversion between the first language and the second language. In this way, the first audio device only needs to send the first voice data, and the second audio device receives the second voice data, and the first audio device and the second audio device need to process fewer programs. Thus, the voice signal processing method can be applied to most audio devices, thereby further improving the applicable scenarios of voice signals, and effectively reducing the power consumption of the first audio device and the second audio device, and improving the duration of voice conversation.
[0164] In some embodiments, Figure 2 As shown, the method also includes:
[0165] S1400: Acquire first information of the first audio device; wherein the first information is at least used to determine a data transmission mode supported by the first audio device;
[0166] S1500: Determine a target data transmission mode of the first wireless connection according to the first information.
[0167] Here, currently there are multiple data transmission modes, and different devices may support different data transmission modes. For example, the data transmission mode may include a first transmission mode, a second transmission mode, and a third transmission mode, wherein the transmission speed of the first transmission mode is higher than the transmission speed of the second transmission mode, and the transmission speed of the second transmission mode is higher than the transmission speed of the third transmission mode. Exemplarily, the first transmission mode may be ultra-wideband UWB (Ultra Wide Band), the second transmission mode may be wireless fidelity WIFI (Wireless Fidelity), and the third transmission mode may be Bluetooth.
[0168] Illustratively, different data transmission modes use different communication chips, follow different communication protocols, and / or use different wireless frequencies.
[0169] According to the first information of the first audio device, the data transmission mode currently suitable for the first audio device can be determined as the target data transmission mode. Exemplarily, when the first information indicates that the resource utilization rate of the current first audio device is low, the first transmission mode can be used for data transmission; when the first information indicates that the resource utilization rate of the current first audio device is high, the third transmission mode can be used. In this way, the target data transmission mode of the first wireless connection is determined through the first information of the first audio device itself, so that the first wireless connection is more stable during the data transmission process, thereby improving the stability of the data transmission of the first wireless connection.
[0170] In another exemplary embodiment, the first information includes a default data transmission mode, which is the data transmission mode used most recently or the data transmission mode used most frequently. For example, the default data transmission mode is Bluetooth, and the target data transmission mode can be set to Bluetooth.
[0171] In the embodiment of the present disclosure, the target data transmission mode of the first wireless connection is determined through the first information of the first audio device, so that the target data transmission mode is more adapted to the first audio device, thereby making the data transmission of the first wireless connection more stable.
[0172] In some embodiments, the first information includes at least one of the following:
[0173] Hardware information of the first audio device; wherein the hardware information indicates the hardware capabilities of the first audio device; different data transmission modes require different hardware capabilities;
[0174] The mode information of the first audio device; the mode information indicates a transmission mode supported by the first audio device.
[0175] The hardware information may be the hardware information of the terminal device or cloud that executes the voice processing method, including at least the processing capability of the processor, whether it has a UWB (Ultra Wide Band) chip, a Bluetooth communication chip or a WIFI (Wireless Fidelity) chip, etc. The processor may include but is not limited to: a central processing unit (CPU), an accelerated processing unit (APU) or a microcontroller unit (MCU), etc.
[0176] In one embodiment, hardware information of the first audio device is obtained, wherein the hardware information is at least used to indicate hardware capabilities of the first audio device, and different data transmission modes require different hardware capabilities. According to the hardware information, a target data transmission mode of the first wireless connection is determined.
[0177] Specifically, the target data transmission mode of the first wireless connection may be determined according to the usage rate of the processor and whether a UWB (Ultra Wide Band) chip, a Bluetooth communication chip or a WIFI (Wireless Fidelity) chip is provided.
[0178] For example, when the usage rate of the processor is lower than the first threshold and the first audio device has a UWB (Ultra Wide Band) chip, ultra wideband (UWB) may be used as the target data transmission mode of the first wireless connection.
[0179] For another example, when the usage rate of the processor is lower than the second threshold, and the first audio device has a WIFI (Wireless Fidelity) chip, Wireless Fidelity WIFI (Wireless Fidelity) can be used as the target data transmission mode of the first wireless connection.
[0180] For another example, when the usage rate of the processor is lower than the third threshold and the first audio device has a Bluetooth chip, Bluetooth may be used as the target data transmission mode of the first wireless connection.
[0181] In the above, the first threshold is smaller than the second threshold, and the second threshold is smaller than the third threshold. The first threshold, the second threshold and the third threshold may be adjusted according to different terminal devices.
[0182] In one embodiment, mode information of the first audio device is obtained. Specifically, mode information sent by the first audio device is received, and the mode information is at least used to indicate a data transmission mode supported by the first audio device. According to the data transmission mode supported by the first audio device, a target data transmission mode of the first wireless connection is determined. The above-mentioned target data transmission mode may be any one of the data transmission modes supported by the first audio device. For example, the target data transmission mode may be a data transmission mode with the highest transmission efficiency and the best transmission stability. In this way, by utilizing the target data transmission mode to transmit the first voice data, the voice conversation is made smoother and more stable.
[0183] The above-mentioned operation information acting on the first preset area of the first audio device can be detected to determine the mode information. The mode information can be all mode information supported by the first audio device, or can be the mode information currently selected by the first audio device. The target data transmission mode is determined by the mode information, so that the current needs of the user or the current situation of the first audio device are fully considered, so that the target data transmission mode can be determined more preparedly.
[0184] For another example, when the operation information indicates a long press operation, the mode information includes all mode information supported by the first audio device. Exemplarily, the mode information includes at least: Ultra Wide Band (UWB), Wireless Fidelity (WIFI), and Bluetooth.
[0185] For example, when the operation information indicates a single-click operation, it is determined that the currently selected mode information is ultra-wideband UWB (Ultra Wide Band).
[0186] For another example, when the operation information indicates a double-click operation, it is determined that the currently selected mode information is Wireless Fidelity WIFI.
[0187] For another example, when the operation information indicates a triple-click operation, the mode information of the current color is determined to be Bluetooth.
[0188] Specifically, the data transmission mode supported by the terminal device can also be determined, and the target data transmission mode of the first wireless connection can be determined according to the data transmission mode supported by the first audio device and the data transmission mode supported by the terminal device. The target data transmission mode here is a data transmission mode supported by both the first audio device and the terminal device, and has the highest transmission efficiency and the best transmission stability.
[0189] For example, the data transmission modes supported by the first audio device include: ultra-wideband UWB (Ultra Wide Band), wireless fidelity WIFI (Wireless Fidelity) and Bluetooth. If the terminal device also supports the above three data transmission modes, the data transmission mode with the highest transmission efficiency and the best transmission stability in the first audio device is determined as the target data transmission mode. The transmission efficiency and transmission stability are ranked from high to low as follows: ultra-wideband UWB (Ultra Wide Band), wireless fidelity WIFI (Wireless Fidelity), Bluetooth. Then, ultra-wideband UWB (Ultra Wide Band) can be used as the target data transmission mode to improve the stability of voice data transmission.
[0190] In some embodiments, the method further comprises:
[0191] The mode information of the target data transmission mode is sent to the first audio device, wherein the target data transmission mode is used for data communication of the first wireless connection.
[0192] The embodiment of the present disclosure sends mode information of the target data transmission mode to the first audio device after determining the target data transmission mode, so that the first audio device performs data communication of the first wireless connection in the target data transmission mode in subsequent voice data communication.
[0193] At present, most devices usually have Bluetooth function. Before the first wireless connection performs data communication in the target data transmission mode, when the target data transmission mode is not Bluetooth, the target data transmission mode can be sent to the first audio device using Bluetooth. When the success message of switching to the target data transmission mode is received from the first audio device, the subsequent first wireless connection performs data communication in the target data transmission mode. When the target data transmission mode is Bluetooth, there is no need to switch the target data transmission mode.
[0194] In some embodiments, Figure 3 As shown, the method also includes:
[0195] S1600: Acquire second information of the second audio device, wherein the second information is at least used to determine a data transmission mode supported by the second audio device;
[0196] S1700: Determine a target data transmission mode of the second wireless connection according to the second information.
[0197] Here, currently, the data transmission modes include multiple types, and different devices may support different data transmission modes. For example, the data transmission modes may include a first transmission mode, a second transmission mode, and a third transmission mode, wherein the transmission speed of the first transmission mode is higher than the transmission speed of the second transmission mode, and the transmission speed of the second transmission mode is higher than the transmission speed of the third transmission mode.
[0198] According to the second information of the second audio device, the data transmission mode currently suitable for the second audio device can be determined as the target data transmission mode. Exemplarily, when the second information indicates that the resource utilization rate of the second audio device is currently low, the first transmission mode can be used for data transmission; when the second information indicates that the resource utilization rate of the second audio device is currently high, the third transmission mode can be used. In this way, the target data transmission mode of the second wireless connection is determined through the second information of the second audio device itself, so that the second wireless connection is more stable during the data transmission process, thereby improving the stability of the data transmission of the second wireless connection.
[0199] In another exemplary embodiment, the second information includes a default data transmission mode, and the target data transmission mode may be set as the default data transmission mode. For example, the default data transmission mode may be the second transmission mode, and the target data transmission mode is set as the second transmission mode.
[0200] In the embodiment of the present disclosure, the target data transmission mode of the second wireless connection is determined through the second information of the second audio device, so that the target data transmission mode is more adapted to the second audio device, thereby making the data transmission of the second wireless connection more stable.
[0201] In one embodiment, the default data transmission mode of the second wireless connection is Bluetooth, and after determining the target data transmission mode, when the target data transmission mode is not Bluetooth, the data transmission mode of the second wireless connection is switched to the target data transmission mode.
[0202] In some embodiments, the second information includes at least one of the following:
[0203] Hardware information of the second audio device; wherein the hardware information indicates the hardware capabilities of the second audio device; different data transmission modes require different hardware capabilities;
[0204] The mode information sent by the second audio device, wherein the mode information indicates a transmission mode supported by the second audio device.
[0205] The hardware information may be the hardware information of the terminal device or cloud that executes the voice processing method, including at least the processing capability of the processor, whether it has a UWB (Ultra Wide Band) chip, a Bluetooth communication chip or a WIFI (Wireless Fidelity) chip, etc. The processor may include but is not limited to: a central processing unit (CPU), an accelerated processing unit (APU) or a microcontroller unit (MCU), etc.
[0206] In one embodiment, hardware information of the second audio device is obtained, wherein the hardware information is at least used to indicate hardware capabilities of the second audio device, and different data transmission modes require different hardware capabilities. According to the hardware information, a target data transmission mode of the second wireless connection is determined.
[0207] Specifically, the target data transmission mode of the second wireless connection may be determined according to the usage rate of the processor and whether a UWB (Ultra Wide Band) chip, a Bluetooth communication chip or a WIFI (Wireless Fidelity) chip is provided.
[0208] For example, when the usage rate of the processor is lower than the first threshold and the second audio device has a UWB (Ultra Wide Band) chip, ultra wideband (UWB) may be used as the target data transmission mode of the second wireless connection.
[0209] For another example, when the usage rate of the processor is lower than the second threshold value and the second audio device has a WIFI (Wireless Fidelity) chip, Wireless Fidelity WIFI (Wireless Fidelity) can be used as the target data transmission mode of the second wireless connection.
[0210] For another example, when the usage rate of the processor is lower than the third threshold and the second audio device has a Bluetooth chip, Bluetooth may be used as the target data transmission mode of the second wireless connection.
[0211] In the above, the first threshold is smaller than the second threshold, and the second threshold is smaller than the third threshold. The first threshold, the second threshold and the third threshold may be adjusted according to different terminal devices.
[0212] In one embodiment, mode information of the second audio device is obtained. Specifically, mode information sent by the second audio device is received, and the mode information is at least used to indicate a data transmission mode supported by the second audio device. According to the data transmission mode supported by the second audio device, a target data transmission mode of the second wireless connection is determined. The above-mentioned target data transmission mode may be any one of the data transmission modes supported by the second audio device. For example, the target data transmission mode may be a data transmission mode with the highest transmission efficiency and the best transmission stability. In this way, by utilizing the target data transmission mode to transmit the second voice data, the voice conversation is made smoother and more stable.
[0213] Specifically, the data transmission mode supported by the terminal device can also be determined, and the target data transmission mode of the second wireless connection can be determined according to the data transmission mode supported by the second audio device and the data transmission mode supported by the terminal device. The target data transmission mode here is a data transmission mode supported by both the second audio device and the terminal device, and has the highest transmission efficiency and the best transmission stability.
[0214] For example, the data transmission modes supported by the second audio device include: ultra-wideband UWB (Ultra Wide Band), wireless fidelity WIFI (Wireless Fidelity) and Bluetooth. If the terminal device also supports the above three data transmission modes, the data transmission mode with the highest transmission efficiency and the best transmission stability in the second audio device is determined as the target data transmission mode. The transmission efficiency and transmission stability are ranked from high to low as follows: ultra-wideband UWB (Ultra Wide Band), wireless fidelity WIFI (Wireless Fidelity), Bluetooth. Then, ultra-wideband UWB (Ultra Wide Band) can be used as the target data transmission mode to improve the stability of voice data transmission.
[0215] In some embodiments, the method further comprises:
[0216] The mode information of the target data transmission mode is sent to the second audio device, wherein the target data transmission mode is used for data communication of the second wireless connection.
[0217] The embodiment of the present disclosure sends mode information of the target data transmission mode to the second audio device after determining the target data transmission mode, so that the second audio device performs data communication of the second wireless connection in the target data transmission mode in subsequent voice data communication.
[0218] At present, most devices usually have Bluetooth function. Before the second wireless connection performs data communication in the target data transmission mode, when the target data transmission mode is not Bluetooth, the target data transmission mode can be sent to the second audio device using Bluetooth. When the success message of switching to the target data transmission mode is received from the second audio device, the subsequent second wireless connection performs data communication in the target data transmission mode. When the target data transmission mode is Bluetooth, there is no need to switch the target data transmission mode.
[0219] In some embodiments, the target data transmission mode includes at least one of the following:
[0220] Ultra-wideband UWB;
[0221] Wireless Fidelity WIFI;
[0222] Bluetooth.
[0223] The above-mentioned ultra-wideband UWB (Ultra Wide Band) is a wireless carrier communication technology or baseband communication technology that uses nanosecond non-sinusoidal narrow pulses to transmit data, so it occupies a wide spectrum range. UWB (Ultra Wide Band) technology has the advantages of low system complexity, low power spectrum density of transmitted signals, insensitivity to channel fading, low interception ability, and high positioning accuracy. It is especially suitable for high-speed wireless access in dense multipath places such as indoors.
[0224] The above-mentioned wireless fidelity WIFI (Wireless Fidelity) has a maximum bandwidth of 11Mbps. In the case of weak signals or interference, the bandwidth can be adjusted to 5.5Mbps, 2Mbps and 1Mbps. The automatic adjustment of bandwidth effectively ensures the stability and reliability of the network. Wireless fidelity WIFI (Wireless Fidelity) has the following advantages: fast speed, high reliability, communication distance of up to 305 meters in open areas, communication distance of 76 meters to 122 meters in closed areas, easy integration with existing wired Ethernet networks, and lower networking costs.
[0225] The above-mentioned Bluetooth is a radio technology that supports short-range communication between devices, and can exchange wireless information between many devices including mobile phones, wireless headphones or laptops. Using Bluetooth technology can effectively simplify the communication between mobile communication terminal devices, so that data transmission becomes faster and more efficient, broadening the way for wireless communication.
[0226] In some embodiments, generating second voice data according to the first voice data includes:
[0227] The first speech data and the language identifier of the second language are input into an intelligent translation model to obtain the second speech data.
[0228] The above intelligent translation mode is an artificial intelligence translation model, which can also be called an AI translation model. The intelligent translation model is a multilingual translation model, which is used for translating voice data, translating first voice data in a first language into second voice data in a second language. The first language and the second language are languages with different language identifiers.
[0229] The intelligent translation model here is for translating between any two languages. The intelligent translation model can automatically identify the language identifier of the first language corresponding to the first voice data.
[0230] For example, when the first voice data is “Life is like a piece of chocolate” and the language identifier of the second language is “ENG-English”, after translation by the intelligent translation model, the second voice data obtained is “Life is like a piece of chocolate”.
[0231] For another example, when the first voice data is “Life is like a box of chocolate” and the language identifier of the second language is “CHN-Chinese”, after translation by the intelligent translation model, the second voice data obtained is “Life is like a box of chocolates”.
[0232] In one embodiment, the intelligent translation model here is an offline translation model, which is on the terminal device. Through the offline translation model, the translation action is directly performed on the terminal device, and no communication with other devices is required to obtain the translated second voice data. This reduces the delay of the translation, improves the translation processing speed, and improves the user experience.
[0233] Of course, the embodiments of the present disclosure do not specifically limit whether the intelligent translation model is an offline translation model, and it can also be an online translation model in the cloud. A suitable intelligent translation model is selected according to the specific situation.
[0234] In some embodiments, Figure 4 As shown, the S1200 includes:
[0235] S1210: Determine whether security verification is required for data transmission over the first wireless connection;
[0236] S1220: When data transmission of the first wireless connection requires security verification, use a security unit to perform security verification on the first voice data;
[0237] S1230: Generate the second voice data according to the first voice data that has passed the security verification.
[0238] The embedded secure element (eSE) here generally refers to the security chip embedded in the terminal. The security element is a general term for components that provide security capabilities, such as encryption and decryption operations, integrity verification, and can securely store sensitive data.
[0239] The above-mentioned security verification includes but is not limited to performing encryption and decryption operations on the first voice data and data integrity verification.
[0240] For example, in private chats or conversations involving business secrets, security verification can be enabled to reduce the possibility of data leakage.
[0241] In one embodiment, a fourth control for whether security verification is required can be set on the call interface, and whether security verification is required is determined based on the operation information acting on the fourth control. The fourth control here can be a selection control, and when security verification is selected, it is determined that the data transmission of the first wireless connection needs security verification, so that the security unit is used to perform security verification on the first voice data.
[0242] In one embodiment, based on the first wireless connection, identification information sent by the first audio device indicating whether security verification is required can be received to determine whether the data transmission of the first wireless connection requires security verification. Specifically, when the identification information sent by the first audio device indicates that security verification is required, the security unit is used to perform security verification on the first voice data, and the second voice data is generated based on the first voice data that has passed the security verification.
[0243] When the data transmission of the first wireless connection requires security verification, the terminal device exchanges keys used in encryption and decryption with the first audio device, so that the voice processing enters the security verification process.
[0244] In one embodiment, the terminal device side may agree to obtain the key after performing a splicing or XOR algorithm based on the Bluetooth MAC address (MAC address) of the first audio device and the Bluetooth MAC address of the terminal device. Since the terminal side knows its own Bluetooth MAC address and the Bluetooth MAC address of the first audio device, it can obtain the key. The first audio device side knows its own Bluetooth MAC address and the Bluetooth MAC address of the terminal device, so it can also obtain the key. This improves the security of data transmission.
[0245] Specifically, when it is determined that the data transmission of the first wireless connection needs security verification, the first voice data is decrypted using the decryption algorithm of the security unit, thereby generating the second voice data based on the decrypted first voice data. The key used in the decryption here is a key that is synchronized and known to each other by the terminal device and the first audio device.
[0246] Here, before or after the first wireless connection is established, at various stages of voice signal processing, when it is detected that the data transmission of the first wireless connection requires security verification, the first voice data is decrypted using the decryption algorithm of the security unit. The disclosed embodiment enhances the confidentiality of the user's call information in a voice call by providing security verification, improves the security of voice processing, and thus improves the user experience.
[0247] In some embodiments, Figure 5 As shown, the S1300 includes:
[0248] S1310: Determine whether security verification is required for data transmission of the second wireless connection;
[0249] S1320: When the data transmission of the second wireless connection requires security verification, use the security unit to perform security verification on the second voice data;
[0250] S1330: Send the second voice data that has passed the security verification to the second audio device.
[0251] The embedded secure element (eSE) here generally refers to the security chip embedded in the terminal. The security element is a general term for components that provide security capabilities, such as encryption and decryption operations, integrity verification, and can securely store sensitive data.
[0252] The above-mentioned security verification includes but is not limited to performing encryption and decryption operations on the second voice data and data integrity verification.
[0253] For example, in private chats or conversations involving business secrets, security verification can be enabled to reduce the possibility of data leakage.
[0254] In one embodiment, a fourth control for whether security verification is required can be set on the call interface, and whether security verification is required is determined based on the operation information acting on the fourth control. The fourth control here can be a selection control, and when security verification is selected, it is determined that the data transmission of the second wireless connection needs security verification, so that the second voice data is security verified.
[0255] In one embodiment, based on the second wireless connection, identification information of whether to perform security verification sent by the second audio device can be received to determine whether the data transmission of the second wireless connection needs security verification. Specifically, when the identification information sent by the second audio device indicates that security verification is required, the security unit is used to perform security verification on the second voice data, so that the second voice data that passes the security verification is sent to the second audio device.
[0256] When the data transmission of the second wireless connection requires security verification, the terminal device exchanges keys used in encryption and decryption with the second audio device, so that the voice processing enters security verification.
[0257] In one embodiment, the terminal device side may agree to obtain the key after performing a splicing or XOR algorithm based on the Bluetooth MAC address (MAC address) of the second audio device and the Bluetooth MAC address of the terminal device. Since the terminal side knows its own Bluetooth MAC address and the Bluetooth MAC address of the second audio device, it can obtain the key. The second audio device side knows its own Bluetooth MAC address and the Bluetooth MAC address of the terminal device, so it can also obtain the key. This improves the security of data transmission.
[0258] Specifically, when it is determined that the data transmission of the second wireless connection needs security verification, the second voice data is encrypted using the encryption algorithm of the security unit, and the encrypted second voice data is sent to the second audio device. The key terminal device and the second audio device used in the encryption here are synchronized and mutually known keys.
[0259] Here, before or after the second wireless connection is established, at various stages of voice signal processing, when it is detected that the data transmission of the second wireless connection requires security verification, the second voice data is encrypted using the encryption algorithm of the security unit. The disclosed embodiment enhances the confidentiality of the user's call information in a voice call by providing security verification, improves the security of voice processing, and thus improves the user experience.
[0260] In some embodiments, before receiving the first voice data sent by the first audio device, the method further includes:
[0261] When an operation acting on a first control on the call interface is detected, the first wireless connection and the second wireless connection are established respectively.
[0262] It can be understood that when a call application (call APP or call applet, etc.) is opened, the call interface can be entered. The first control on the call interface is a call start control. When the first control is clicked, the first wireless connection and the second wireless connection are established. Here, the first wireless connection and the second wireless connection can be Bluetooth connections by default. When the first control is clicked again, the first wireless connection and the second wireless connection are disconnected.
[0263] After establishing a first wireless connection with the first audio device and establishing a second wireless connection with the second audio device, relevant information of the currently connected first audio device and the second audio device is displayed on the call interface.
[0264] Here, nicknames corresponding to the first audio device and the second audio device can also be set on the call interface. The call interface can be projected on any screen (for example, a TV screen). By setting the nickname, it is convenient to determine the current speaker.
[0265] In some embodiments, the method further comprises:
[0266] When an operation acting on a second control on the call interface is detected, text information of the first voice data and the second voice data is displayed.
[0267] The second control is used to identify whether to display the text information of the first voice data and the second voice data. The second control is a selection control, and when the text information is selected, the text information of the first voice data and the second voice data is displayed in the first area of the call interface.
[0268] The text information here may include first voice data in a first language and second voice data in a second language. The nickname of the user who sent the first voice data, that is, the nickname corresponding to the set first audio device, can also be displayed in the text information, so as to facilitate determining which user sent the current text information.
[0269] The text information in the embodiment of the present disclosure can be displayed in real time on the screen in the form of text during a call, and the text information in the first language and the text information in the second language can be displayed separately, so that the user can view the text information of the call conveniently and it is more suitable for conference mode.
[0270] In one embodiment, the call interface also has a fifth control. When an operation on the fifth control on the call interface is detected, the user is prompted to save the text and a language selection control for outputting the text is displayed, so that the user can select the language identifier used for outputting the text. Thus, when outputting text information, text information in different languages can be output according to the user's selection. For example, when the language identifier set by the user is Chinese, Chinese text information is output, and when the language identifier set by the user is English, English text information is output.
[0271] The disclosed embodiment may include multiple first audio devices, and different first audio devices may be set with different nicknames, so that the text information is clearer and easier for the user to view or organize the meeting after the call ends.
[0272] In some embodiments, the method further comprises:
[0273] Displaying device information of the second audio device on the language setting interface;
[0274] When an operation acting on a third control on the language setting interface is detected, obtaining operation information based on the third control;
[0275] A language identifier of the second language of the second audio device is determined according to the operation information.
[0276] It is understandable that there is an entrance to the language setting interface on the call interface, and after entering the language setting interface, the device information of the second audio device currently establishing the second wireless connection is displayed, wherein the device information at least includes the brand, model and nickname of the second audio device, etc. By displaying the device information, it is convenient for the user to determine the target second audio device before setting the language identifier of the second language.
[0277] Here, when the third control acting on the position of the device information of the second audio device on the language setting interface is detected, the language identifier of the second language is selected, and the language identifiers of all supported second languages can be displayed here to facilitate user selection.
[0278] The operation information of the third control is to select the language identifier of the second language, and the selected language identifier is determined to be the language identifier of the second language of the second audio device.
[0279] The device information of all second audio devices with which the second wireless connection is established is displayed on the language setting interface, and the operation of the third control mentioned above can be performed based on the selected second audio device, so as to facilitate the user to set the language identification of the second language of each second audio device. In order to lay the foundation for realizing that multiple second audio devices output voice data in different second languages, the embodiment of the present disclosure can set the language identification at any stage of voice signal processing, thus providing convenience for user operation.
[0280] For example, if the user wearing the second audio device changes, and the language proficient in the new user is different from that of the previous user, the corresponding second language can be reset through the language setting interface, thereby facilitating the user to have a voice conversation and improving the user experience.
[0281] In one embodiment, the method further includes: based on the second wireless connection with the second audio device, receiving third voice data sent by the second audio device, wherein the third voice data is voice data in the second language; generating fourth voice data according to the third voice data, wherein the fourth voice data is voice data in the first language; based on the first wireless connection with the first audio device, sending the fourth voice data to the first audio device, wherein the fourth voice data is at least used for output of the first audio device. In this way, a voice call function between the first audio device and the second device is implemented.
[0282] like Figure 6 As shown, an embodiment of the present disclosure provides a voice signal processing method, which is applied to a first audio device, and the method includes:
[0283] S2100: Sending mode information based on the first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device;
[0284] S2200: Receive mode information of a target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send first voice data in the target transmission mode, and the end having the first wireless connection with the first audio device receives the first voice data in the target transmission mode, generates second voice data based on the first voice data, and sends the second voice data to at least one second audio device.
[0285] The first audio device in the embodiment of the present disclosure may be any audio device with a wireless communication function. For example, the first audio device includes but is not limited to a wireless headset, a speaker, or a microphone.
[0286] The above-mentioned mode information is the transmission mode supported by the first audio device. Exemplarily, the transmission mode supported by the first audio device includes but is not limited to short-range communication networks such as wireless fidelity WIFI (Wireless Fidelity), Bluetooth, Zigbee, ultra-wideband UWB (Ultra Wide Band) or near field communication (near field communication, NFC) network.
[0287] The mode information of the transmission mode supported by the first audio device may be determined by touching. Specifically, the mode information may be determined according to the operation information acting on the first preset area of the first audio device.
[0288] For example, when the operation information indicates a single-click operation, the mode information at least includes ultra-wideband (UWB).
[0289] For another example, when the operation information indicates a double-click operation, the mode information at least includes Wireless Fidelity WIFI.
[0290] For another example, when the operation information indicates a triple-click operation, the mode information at least includes: Bluetooth.
[0291] For another example, when the operation information indicates a long press operation, the mode information at least includes: Ultra Wide Band (UWB), Wireless Fidelity (WIFI), and Bluetooth.
[0292] The device that establishes the first wireless connection with the first audio device can be any terminal device with a wireless connection or a short-range connection function. The terminal device may include various electronic devices, servers or clouds with wireless communication functions. The electronic devices here may include mobile terminals and fixed terminals. Exemplarily, mobile terminals include but are not limited to smart phones, smart TVs, tablet computers, laptops, smart cars or wearable devices (e.g., smart watches, etc.). Exemplarily, fixed terminals include but are not limited to desktop computers, etc.
[0293] In some embodiments, the target data transmission mode includes at least one of the following:
[0294] Ultra-wideband UWB;
[0295] Wireless Fidelity WIFI;
[0296] Bluetooth.
[0297] The above-mentioned ultra-wideband UWB (Ultra Wide Band) is a wireless carrier communication technology or baseband communication technology that uses nanosecond non-sinusoidal narrow pulses to transmit data, so it occupies a wide spectrum range. UWB (Ultra Wide Band) technology has the advantages of low system complexity, low power spectrum density of transmitted signals, insensitivity to channel fading, low interception ability, and high positioning accuracy. It is especially suitable for high-speed wireless access in dense multipath places such as indoors.
[0298] The above-mentioned wireless fidelity WIFI (Wireless Fidelity) has a maximum bandwidth of 11Mbps. In the case of weak signals or interference, the bandwidth can be adjusted to 5.5Mbps, 2Mbps and 1Mbps. The automatic adjustment of bandwidth effectively ensures the stability and reliability of the network. Wireless fidelity WIFI (Wireless Fidelity) has the following advantages: fast speed, high reliability, communication distance of up to 305 meters in open areas, communication distance of 76 meters to 122 meters in closed areas, easy integration with existing wired Ethernet networks, and lower networking costs.
[0299] The above-mentioned Bluetooth is a radio technology that supports short-range communication between devices, and can exchange wireless information between many devices including mobile phones, wireless headphones or laptops. Using Bluetooth technology can effectively simplify the communication between mobile communication terminal devices, so that data transmission becomes faster and more efficient, broadening the way for wireless communication.
[0300] In some embodiments, sending the first voice data collected by the first audio device based on the first wireless connection includes:
[0301] Determining whether security verification is required for data transmission over the first wireless connection;
[0302] When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data;
[0303] The first voice data after security verification is sent.
[0304] The embedded secure element (eSE) here generally refers to a security chip embedded in the first audio device. The security element is a general term for components that provide security capabilities, such as encryption and decryption operations, integrity verification, and can securely store sensitive data.
[0305] The above-mentioned security verification includes but is not limited to performing encryption and decryption operations on the second voice data and data integrity verification.
[0306] For example, in private chats or conversations involving business secrets, security verification can be enabled to reduce the possibility of data leakage.
[0307] For example, when a single-click operation is performed in the second preset area of the first audio device, it is determined that the first wireless connection needs to be security verified.
[0308] For another example, when a double-click operation is performed in the second preset area of the first audio device, it is determined that the first wireless connection does not require security verification.
[0309] In one embodiment, when a single-click operation is performed in the second preset area of the first audio device based on the first wireless connection, identification information indicating whether security verification is required can be sent. Specifically, when the end that has established the first wireless connection with the first audio device receives the identification information indicating whether security verification is required sent by the first audio device and indicates to perform security verification, the end uses the decryption algorithm of the security unit to decrypt the first voice data, and generates the second voice data based on the first voice data.
[0310] The disclosed embodiments provide security verification to enhance the confidentiality of the user's call information during voice calls, thereby improving the security of voice processing and thus enhancing the user experience.
[0311] In one embodiment, when the first wireless connection needs to be verified for security, the first voice data is encrypted using an encryption algorithm in a security unit of the first audio device. Alternatively, when the first wireless connection needs to be verified for security, the first voice data can be encrypted using a key synchronized to the first audio device at one end with which the first wireless connection is established, thereby obtaining encrypted first voice data.
[0312] like Figure 7 As shown, an embodiment of the present disclosure provides a voice signal processing method, which is applied to a second audio device, and the method includes:
[0313] S3100: Sending mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device;
[0314] S3200: Receive mode information of a target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data;
[0315] The second voice data is generated by receiving the first voice data sent by the first audio device at one end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and generating the second voice data according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
[0316] The second audio device in the embodiment of the present disclosure may be any audio device with a wireless communication function. For example, the second audio device includes but is not limited to a wireless headset, a speaker, or a microphone.
[0317] The above-mentioned mode information is the transmission mode supported by the second audio device. Exemplarily, the transmission mode supported by the second audio device includes but is not limited to short-range communication networks such as wireless fidelity WIFI (Wireless Fidelity), Bluetooth, Zigbee, ultra-wideband UWB (Ultra Wide Band) or near field communication (near field communication, NFC) network.
[0318] The mode information of the transmission mode supported by the second audio device may be determined by touching. Specifically, the mode information may be determined based on the operation information detected acting on the preset area of the second audio device.
[0319] For example, when the operation information indicates a single-click operation, the mode information at least includes ultra-wideband (UWB).
[0320] For another example, when the operation information indicates a double-click operation, the mode information at least includes Wireless Fidelity WIFI.
[0321] For another example, when the operation information indicates a triple-click operation, the mode information at least includes: Bluetooth.
[0322] For another example, when the operation information indicates a long press operation, the mode information at least includes: Ultra Wide Band (UWB), Wireless Fidelity (WIFI), and Bluetooth.
[0323] The device for establishing the second wireless connection with the second audio device can be any terminal device with wireless connection or short-range connection function. The terminal device may include various electronic devices, servers or clouds with wireless communication functions. The electronic devices here may include mobile terminals and fixed terminals. Exemplarily, mobile terminals include but are not limited to smart phones, smart TVs, tablet computers, laptops, smart cars or wearable devices (e.g., smart watches, etc.). Exemplarily, fixed terminals include but are not limited to desktop computers, etc.
[0324] In some embodiments, the target data transmission mode includes at least one of the following:
[0325] Ultra-wideband UWB;
[0326] Wireless Fidelity WIFI;
[0327] Bluetooth.
[0328] The above-mentioned ultra-wideband UWB (Ultra Wide Band) is a wireless carrier communication technology or baseband communication technology that uses nanosecond non-sinusoidal narrow pulses to transmit data, so it occupies a wide spectrum range. UWB (Ultra Wide Band) technology has the advantages of low system complexity, low power spectrum density of transmitted signals, insensitivity to channel fading, low interception ability, and high positioning accuracy. It is especially suitable for high-speed wireless access in dense multipath places such as indoors.
[0329] The above-mentioned wireless fidelity WIFI (Wireless Fidelity) has a maximum bandwidth of 11Mbps. In the case of weak signals or interference, the bandwidth can be adjusted to 5.5Mbps, 2Mbps and 1Mbps. The automatic adjustment of bandwidth effectively ensures the stability and reliability of the network. Wireless fidelity WIFI (Wireless Fidelity) has the following advantages: fast speed, high reliability, communication distance of up to 305 meters in open areas, communication distance of 76 meters to 122 meters in closed areas, easy integration with existing wired Ethernet networks, and lower networking costs.
[0330] The above-mentioned Bluetooth is a radio technology that supports short-range communication between devices, and can exchange wireless information between many devices including mobile phones, wireless headphones or laptops. Using Bluetooth technology can effectively simplify the communication between mobile communication terminal devices, so that data transmission becomes faster and more efficient, broadening the way for wireless communication.
[0331] In some embodiments, the method further comprises:
[0332] determining whether security verification is required for data transmission of the second wireless connection;
[0333] When the data transmission of the second wireless connection requires security verification, the security unit is used to perform security verification on the second voice data.
[0334] The embedded secure element (eSE) here generally refers to a security chip embedded in the second audio device. The security element is a general term for components that provide security capabilities, such as encryption and decryption operations, integrity verification, and can securely store sensitive data.
[0335] The above-mentioned security verification includes but is not limited to performing encryption and decryption operations on the second voice data and data integrity verification.
[0336] For example, in private chats or conversations involving business secrets, security verification can be enabled to reduce the possibility of data leakage.
[0337] For example, when a single-click operation is performed in the third preset area of the second audio device, it is determined that the second wireless connection requires security verification.
[0338] For another example, when a double-click operation is performed in the third preset area of the second audio device, it is determined that the second wireless connection does not require security verification.
[0339] In one embodiment, when a single-click operation is performed in the third preset area of the second audio device based on the second wireless connection, identification information indicating whether security verification is required is sent. Specifically, when the end that has established the second wireless connection with the second audio device receives the identification information indicating whether security verification is required sent by the second audio device and indicates that security verification is required, the second voice data is encrypting using the decryption algorithm of the security unit.
[0340] The disclosed embodiments provide security verification to enhance the confidentiality of the user's call information during voice calls, thereby improving the security of voice processing and thus enhancing the user experience.
[0341] In one embodiment, when the second wireless connection needs to be verified for security, the second voice data is decrypted using a decryption algorithm in a security unit of the second audio device. Alternatively, when the second wireless connection needs to be verified for security, the second voice data is decrypted using a key synchronized to the second audio device at one end with which the second wireless connection is established, thereby obtaining decrypted second voice data.
[0342] The present disclosure provides a method for voice processing using Figure 8 The voice processing method is performed by the smart terminal, the first audio device, and the second audio device shown.
[0343] Exemplarily, both the first audio device and the second audio device may be true wireless stereo headsets (True Wireless Stereo, TWS).
[0344] The earphones (i.e., the first audio device and the second audio device) in the disclosed embodiment have, in addition to Bluetooth and an audio unit (Audio), a low-power wireless fidelity WIFI (Wireless Fidelity) chip, and may also include an ultra-wideband UWB (Ultra Wide Band) chip.
[0345] The headset (i.e., the first audio device and the second audio device) transmits the received information (i.e., the first voice data mentioned above) to the smart terminal (such as a smart phone). The transmission mode can be divided into three gears: confidentiality mode, i.e., ultra-wideband UWB (Ultra WideBand) and eSE security unit, high-frequency anti-interference mode (WIFI 5 / 6G) and low-power mode (BT 2.4G or WiFi2.4G). Among them, WIFI5 / 6G refers to WIFI (Wireless Fidelity) with a frequency of 5GHz and WIFI (Wireless Fidelity) with a frequency of 6GHZ. BT2.4G refers to Bluetooth with a frequency of 2.4GHZ.
[0346] The intelligent terminal completes the language translation through the AI translation model receiving the sound information, that is, completes the processing of converting the first voice data into the second voice data. Among them, the intelligent terminal can perform translation processing through the main processor as an accelerated processor (Accelerated Processing Units, APU) or an AI accelerator (Neural Processor Unit, NPU).
[0347] Depending on the processing capabilities of the smart terminal, it can be translated into one language or multiple languages (such as English into Chinese, Japanese or Korean, etc.).
[0348] The smart terminal synchronizes the translated language content to the target headset (the confidentiality, high frequency and low power consumption modes need to be set according to the hardware conditions of the peer device). The smart terminal can synchronize the content of different languages to different headsets through the heterogeneous mode, such as Figure 8 headphone A and headphone B.
[0349] like Fig. 9 As shown, the embodiment of the present disclosure provides a speech signal processing device 400, the device comprising:
[0350] A first receiving module 410 is configured to receive first voice data sent by a first audio device based on a first wireless connection with the first audio device; wherein the first voice data is voice data in a first language;
[0351] A generating module 420, configured to generate second voice data according to the first voice data; wherein the second voice data is voice data in a second language;
[0352] The first sending module 430 is configured to send the second voice data to at least one second audio device based on at least one second wireless connection with the second audio device, wherein the second voice data is at least output by the second audio device.
[0353] In some embodiments, the apparatus further comprises:
[0354] A first acquisition module, configured to acquire first information of the first audio device; wherein the first information is at least used to determine a data transmission mode supported by the first audio device;
[0355] A first determining module is configured to determine a target data transmission mode of the first wireless connection according to the first information.
[0356] In some embodiments, the first information includes at least one of the following:
[0357] Hardware information of the first audio device; wherein the hardware information indicates the hardware capabilities of the first audio device; different data transmission modes require different hardware capabilities;
[0358] The mode information of the first audio device; the mode information indicates a transmission mode supported by the first audio device.
[0359] In some embodiments, the apparatus further comprises:
[0360] The second receiving module is used to send mode information of the target data transmission mode to the first audio device, wherein the target data transmission mode is used for data communication of the first wireless connection.
[0361] In some embodiments, the apparatus further comprises:
[0362] A second acquisition module, configured to acquire second information of the second audio device, wherein the second information is at least used to determine a data transmission mode supported by the second audio device;
[0363] A second determining module is configured to determine a target data transmission mode of the second wireless connection according to the second information.
[0364] In some embodiments, the second information includes at least one of the following:
[0365] Hardware information of the second audio device; wherein the hardware information indicates the hardware capabilities of the second audio device; different data transmission modes require different hardware capabilities;
[0366] The mode information sent by the second audio device, wherein the mode information indicates a transmission mode supported by the second audio device.
[0367] In some embodiments, the apparatus further comprises:
[0368] The third sending module is used to send mode information of the target data transmission mode to the second audio device, wherein the target data transmission mode is used for data communication of the second wireless connection.
[0369] In some embodiments, the target data transmission mode includes at least one of the following:
[0370] Ultra-wideband UWB;
[0371] Wireless Fidelity WIFI;
[0372] Bluetooth.
[0373] In some embodiments, the generating module is specifically configured to input the first speech data and the language identifier of the second language into an intelligent translation model to obtain the second speech data.
[0374] In some embodiments, the generation module is specifically used to determine whether the data transmission of the first wireless connection requires security verification; when the data transmission of the first wireless connection requires security verification, use the security unit to perform security verification on the first voice data; and generate the second voice data based on the first voice data that passes the security verification.
[0375] In some embodiments, the first sending module is specifically used to determine whether the data transmission of the second wireless connection requires security verification; when the data transmission of the second wireless connection requires security verification, the second voice data is security verified using the security unit; and the second voice data that passes the security verification is sent to the second audio device.
[0376] In some embodiments, the apparatus further comprises:
[0377] The connection establishment module is used to establish the first wireless connection and the second wireless connection respectively when an operation acting on the first control on the call interface is detected.
[0378] In some embodiments, the apparatus further comprises:
[0379] The first display module is used to display text information of the first voice data and the second voice data when an operation acting on the second control on the call interface is detected.
[0380] In some embodiments, the apparatus further comprises:
[0381] A second display module, used for displaying the device information of the second audio device on the language setting interface;
[0382] A third acquisition module, configured to acquire operation information based on the third control when an operation acting on the third control on the language setting interface is detected;
[0383] A third determining module is configured to determine a language identifier of the second language of the second audio device according to the operation information.
[0384] like Fig.10 As shown, the embodiment of the present disclosure provides a speech signal processing device 500, which is applied to a first audio device, and the device includes:
[0385] A fourth sending module 510, configured to send mode information based on the first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device;
[0386] The second receiving module 520 is used to receive mode information of the target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send the first voice data in the target transmission mode, the end having the first wireless connection with the first audio device receives the first voice data in the target transmission mode, generates second voice data based on the first voice data, and sends the second voice data to at least one second audio device.
[0387] In some embodiments, the target data transmission mode includes at least one of the following:
[0388] Ultra-wideband UWB;
[0389] Wireless Fidelity WIFI;
[0390] Bluetooth.
[0391] In some embodiments, the fourth sending module is specifically used to determine whether the data transmission of the first wireless connection requires security verification; when the data transmission of the first wireless connection requires security verification, use the security unit to perform security verification on the first voice data; and send the first voice data after security verification.
[0392] like Fig.11 As shown, the embodiment of the present disclosure provides a speech signal processing device 600, which is applied to a second audio device, and the device includes:
[0393] A fifth sending module 610, configured to send mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device;
[0394] The third receiving module 620 is used to receive mode information of the target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data; the second voice data is the first voice data sent by the first audio device received by the end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and the second voice data is generated according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
[0395] In some embodiments, the target data transmission mode includes at least one of the following:
[0396] Ultra-wideband UWB;
[0397] Wireless Fidelity WIFI;
[0398] Bluetooth.
[0399] In some embodiments, the apparatus further comprises:
[0400] A fourth determination module, used to determine whether the data transmission of the second wireless connection requires security verification;
[0401] The security verification module is used to perform security verification on the second voice data using the security unit when the data transmission of the second wireless connection needs to be security verified.
[0402] like Fig.12 As shown, an embodiment of the present disclosure provides a terminal device, including:
[0403] a memory for storing processor-executable instructions;
[0404] A processor connected to the memory;
[0405] Among them, the processor is configured to execute the speech signal processing method provided by any of the aforementioned technical solutions.
[0406] The processor may include various types of storage media, which are non-transitory computer storage media that can continue to memorize information stored thereon after the terminal device loses power.
[0407] The processor can be connected to the memory through a bus or the like, and is used to read the executable program stored in the memory, for example, to execute the following Figures 1 to 7 At least one of any of the methods shown.
[0408] Fig.12 8 is a block diagram of a terminal device 800 according to an exemplary embodiment. For example, the terminal device 800 may be included in a terminal device such as a mobile phone, a mobile computer, or a server, etc. In short, the data processing terminal device 800 may be included in any terminal device.
[0409] Reference Fig.12 The terminal device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .
[0410] The processing component 802 generally controls the overall operation of the terminal device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
[0411] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the terminal device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0412] The power supply component 806 provides power to various components of the terminal device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 800.
[0413] The multimedia component 808 includes a screen that provides an output interface between the terminal device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating state, such as a shooting state or a video state, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0414] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the terminal device 800 is in an operating state, such as a call state, a recording state, and a voice recognition state, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
[0415] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0416] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the terminal device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of the components, such as the display and keypad of the terminal device 800, and the sensor assembly 814 can also detect the position change of the terminal device 800 or a component of the terminal device 800, the presence or absence of contact between the user and the terminal device 800, the orientation or acceleration / deceleration of the terminal device 800, and the temperature change of the terminal device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0417] The communication component 816 is configured to facilitate the communication between the terminal device 800 and other devices in a wired or wireless manner. The terminal device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0418] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0419] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0420] The present disclosure provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the UE, the UE or the base station can execute the speech signal processing method provided in any of the above embodiments, and can execute Figures 1 to 7 At least one of any of the methods shown.
[0421] A voice signal processing method may include: based on a first wireless connection with a first audio device, receiving first voice data sent by the first audio device; wherein the first voice data is voice data in a first language; generating second voice data based on the first voice data; wherein the second voice data is voice data in a second language; based on at least one second wireless connection with a second audio device, sending the second voice data to at least one second audio device, wherein the second voice data is at least used for output by the second audio device.
[0422] It is understandable that the method further comprises:
[0423] Acquire first information of the first audio device; wherein the first information is at least used to determine a data transmission mode supported by the first audio device;
[0424] A target data transmission mode of the first wireless connection is determined according to the first information.
[0425] It can be understood that the first information includes at least one of the following:
[0426] Hardware information of the first audio device; wherein the hardware information indicates the hardware capabilities of the first audio device; different data transmission modes require different hardware capabilities;
[0427] The mode information of the first audio device; the mode information indicates a transmission mode supported by the first audio device.
[0428] It can be understood that the method further includes:
[0429] The mode information of the target data transmission mode is sent to the first audio device, wherein the target data transmission mode is used for data communication of the first wireless connection.
[0430] It is understandable that the method further comprises:
[0431] Acquire second information of the second audio device, wherein the second information is at least used to determine a data transmission mode supported by the second audio device;
[0432] A target data transmission mode of the second wireless connection is determined according to the second information.
[0433] It can be understood that the second information includes at least one of the following:
[0434] Hardware information of the second audio device; wherein the hardware information indicates the hardware capabilities of the second audio device; different data transmission modes require different hardware capabilities;
[0435] The mode information sent by the second audio device, wherein the mode information indicates a transmission mode supported by the second audio device.
[0436] It is understandable that the method further comprises:
[0437] The mode information of the target data transmission mode is sent to the second audio device, wherein the target data transmission mode is used for data communication of the second wireless connection.
[0438] It can be understood that the target data transmission mode includes at least one of the following:
[0439] Ultra-wideband UWB;
[0440] Wireless Fidelity WIFI;
[0441] Bluetooth.
[0442] It can be understood that the generating of the second voice data according to the first voice data includes:
[0443] The first speech data and the language identifier of the second language are input into an intelligent translation model to obtain the second speech data.
[0444] It can be understood that the generating of the second voice data according to the first voice data includes:
[0445] Determining whether security verification is required for data transmission over the first wireless connection;
[0446] When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data;
[0447] The second voice data is generated according to the first voice data that has passed the security verification.
[0448] It can be understood that, based on at least one second wireless connection with a second audio device, sending the second voice data to at least one second audio device includes:
[0449] determining whether security verification is required for data transmission of the second wireless connection;
[0450] When the data transmission of the second wireless connection requires security verification, using the security unit to perform security verification on the second voice data;
[0451] The second voice data that passes the security verification is sent to the second audio device.
[0452] It can be understood that before receiving the first voice data sent by the first audio device, the method further includes:
[0453] When an operation acting on a first control on the call interface is detected, the first wireless connection and the second wireless connection are established respectively.
[0454] It is understandable that the method further comprises:
[0455] When an operation acting on a second control on the call interface is detected, text information of the first voice data and the second voice data is displayed.
[0456] It is understandable that the method further comprises:
[0457] Displaying device information of the second audio device on the language setting interface;
[0458] When an operation acting on a third control on the language setting interface is detected, obtaining operation information based on the third control;
[0459] A language identifier of the second language of the second audio device is determined according to the operation information.
[0460] A voice signal processing method applied to a first audio device, the method comprising: sending mode information based on a first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device; receiving mode information of a target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send first voice data in the target transmission mode, an end having the first wireless connection with the first audio device receives the first voice data in the target transmission mode, generates second voice data based on the first voice data, and sends the second voice data to at least one second audio device.
[0461] It can be understood that the target data transmission mode includes at least one of the following:
[0462] Ultra-wideband UWB;
[0463] Wireless Fidelity WIFI;
[0464] Bluetooth.
[0465] It can be understood that, the sending, based on the first wireless connection, the first voice data collected by the first audio device includes:
[0466] Determining whether security verification is required for data transmission over the first wireless connection;
[0467] When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data;
[0468] The first voice data after security verification is sent.
[0469] A speech signal processing method applied to a second audio device, the method comprising:
[0470] Sending mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device;
[0471] receiving mode information of a target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data;
[0472] The second voice data is generated by receiving the first voice data sent by the first audio device at one end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and generating the second voice data according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
[0473] It can be understood that the target data transmission mode includes at least one of the following:
[0474] Ultra-wideband UWB;
[0475] Wireless Fidelity WIFI;
[0476] Bluetooth.
[0477] It is understandable that the method further comprises:
[0478] determining whether security verification is required for data transmission of the second wireless connection;
[0479] When the data transmission of the second wireless connection requires security verification, the security unit is used to perform security verification on the second voice data.
[0480] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0481] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A speech signal processing method, characterized in that: The method comprises: Based on a first wireless connection with a first audio device, receiving first voice data sent by the first audio device; wherein the first voice data is voice data in a first language; Generate second voice data according to the first voice data; wherein the second voice data is voice data in a second language; Based on at least one second wireless connection with a second audio device, the second voice data is sent to at least one second audio device, wherein the second voice data is at least used for output by the second audio device.
2. The method according to claim 1, characterized in that: The method further comprises: Acquire first information of the first audio device; wherein the first information is at least used to determine a data transmission mode supported by the first audio device; A target data transmission mode of the first wireless connection is determined according to the first information.
3. The method according to claim 2, characterized in that The first information includes at least one of the following: Hardware information of the first audio device; wherein the hardware information indicates the hardware capabilities of the first audio device; different data transmission modes require different hardware capabilities; The mode information of the first audio device; the mode information indicates a transmission mode supported by the first audio device.
4. The method according to claim 2, characterized in that: The method further comprises: The mode information of the target data transmission mode is sent to the first audio device, wherein the target data transmission mode is used for data communication of the first wireless connection.
5. The method according to claim 1, characterized in that The method further comprises: Acquire second information of the second audio device, wherein the second information is at least used to determine a data transmission mode supported by the second audio device; A target data transmission mode of the second wireless connection is determined according to the second information.
6. The method according to claim 5, characterized in that The second information includes at least one of the following: Hardware information of the second audio device; wherein the hardware information indicates the hardware capabilities of the second audio device; different data transmission modes require different hardware capabilities; The mode information sent by the second audio device, wherein the mode information indicates a transmission mode supported by the second audio device.
7. The method according to claim 5, characterized in that The method further comprises: The mode information of the target data transmission mode is sent to the second audio device, wherein the target data transmission mode is used for data communication of the second wireless connection.
8. The method according to any one of claims 2 to 7, characterized in that: The target data transmission mode includes at least one of the following: Ultra-wideband UWB; Wireless Fidelity WIFI; Bluetooth.
9. The method according to claim 1, characterized in that: The step of generating second voice data according to the first voice data includes: The first speech data and the language identifier of the second language are input into an intelligent translation model to obtain the second speech data.
10. The method according to claim 1, characterized in that The step of generating second voice data according to the first voice data includes: Determining whether security verification is required for data transmission over the first wireless connection; When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data; The second voice data is generated according to the first voice data that has passed the security verification.
11. The method according to claim 1, characterized in that: The sending the second voice data to at least one second audio device based on at least one second wireless connection with the second audio device comprises: determining whether security verification is required for data transmission of the second wireless connection; When the data transmission of the second wireless connection requires security verification, using the security unit to perform security verification on the second voice data; The second voice data that passes the security verification is sent to the second audio device.
12. The method according to claim 1, characterized in that Before receiving the first voice data sent by the first audio device, the method further includes: When an operation acting on a first control on the call interface is detected, the first wireless connection and the second wireless connection are established respectively.
13. The method according to claim 12, characterized in that The method further comprises: When an operation acting on a second control on the call interface is detected, text information of the first voice data and the second voice data is displayed.
14. The method according to claim 1, characterized in that The method further comprises: Displaying device information of the second audio device on the language setting interface; When an operation acting on a third control on the language setting interface is detected, obtaining operation information based on the third control; A language identifier of the second language of the second audio device is determined according to the operation information.
15. A speech signal processing method, characterized in that: Applied to a first audio device, the method comprises: Sending mode information based on the first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device; Receive mode information of a target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send first voice data in the target transmission mode, an end having the first wireless connection with the first audio device receives the first voice data in the target transmission mode, generates second voice data based on the first voice data, and sends the second voice data to at least one second audio device.
16. The method according to claim 15, characterized in that The target data transmission mode includes at least one of the following: Ultra-wideband UWB; Wireless Fidelity WIFI; Bluetooth.
17. The method according to claim 15, characterized in that The sending, based on the first wireless connection, the first voice data collected by the first audio device, includes: Determining whether security verification is required for data transmission over the first wireless connection; When the data transmission of the first wireless connection requires security verification, using the security unit to perform security verification on the first voice data; The first voice data after security verification is sent.
18. A speech signal processing method, characterized in that: Applied to a second audio device, the method comprises: Sending mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device; receiving mode information of a target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data; The second voice data is generated by receiving the first voice data sent by the first audio device at one end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and generating the second voice data according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
19. The method according to claim 18, characterized in that The target data transmission mode includes at least one of the following: Ultra-wideband UWB; Wireless Fidelity WIFI; Bluetooth.
20. The method according to claim 18, characterized in that The method further comprises: determining whether security verification is required for data transmission of the second wireless connection; When the data transmission of the second wireless connection requires security verification, the security unit is used to perform security verification on the second voice data.
21. A speech signal processing device, the device comprising: A first receiving module, configured to receive first voice data sent by a first audio device based on a first wireless connection with the first audio device; wherein the first voice data is voice data in a first language; A generating module, configured to generate second voice data according to the first voice data; wherein the second voice data is voice data in a second language; The first sending module is configured to send the second voice data to at least one second audio device based on at least one second wireless connection with the second audio device, wherein the second voice data is at least used for output of the second audio device.
22. A speech signal processing device, applied to a first audio device, the device comprising: a fourth sending module, configured to send mode information based on the first wireless connection, wherein the mode information at least includes a transmission mode supported by the first audio device; A second receiving module is used to receive mode information of a target transmission mode returned based on the mode information, wherein the target transmission mode is used for the first audio device to send first voice data in the target transmission mode, an end having the first wireless connection with the first audio device to receive the first voice data in the target transmission mode, generate second voice data based on the first voice data, and send the second voice data to at least one second audio device.
23. A speech signal processing device, applied to a second audio device, the device comprising: a fifth sending module, configured to send mode information based on the second wireless connection, wherein the mode information at least includes a transmission mode supported by the second audio device; A third receiving module is used to receive mode information of a target data transmission mode returned based on the mode information, wherein the target data transmission mode is used for the second audio device to receive the second voice data in the target transmission mode so that the second audio device outputs the second voice data; the second voice data is the first voice data sent by the first audio device received by the end having the second wireless connection with the second audio device based on the first wireless connection with the first audio device, and the second voice data is generated according to the first voice data; the first voice data is voice data in a first language, and the second voice data is voice data in a second language.
24. A terminal device, characterized in that: include: a memory for storing processor-executable instructions; A processor connected to the memory; The processor is configured to execute the speech signal processing method provided in any one of claims 1 to 14 or 15 to 17 or 18 to 20.
25. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a computer, enables the computer to perform the speech signal processing method provided in any one of claims 1 to 14 or 15 to 17 or 18 to 20.
