Call method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a call method and device, electronic equipment and a readable storage medium, and belongs to the technical field of communication. The method comprises the steps of receiving first voice information sent by second electronic equipment under the condition that first electronic equipment communicates with the second electronic equipment; converting the first voice information into first text information, and displaying the first text information; obtaining second text information input by the user; and converting the second text information into second voice information based on the first tone, and sending the second voice information to second electronic equipment.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to a call method, device, electronic device and readable storage medium. Background Art

[0002] With the development of mobile communication technology, voice calls have become one of the main functions of electronic devices such as mobile phones, telephones, and walkie-talkies. Voice calls are an efficient and convenient communication method for real-time communication using voice as a medium. In daily life, callers and receivers often need real-time voice calls.

[0003] Existing voice calls use the "voice-to-voice" method, using long connections to transmit voice data streams in real time. In some "immersive" scenarios in daily life and work, for example, in public places such as conference rooms, office areas, libraries, and classrooms, when the receiver receives a voice call from the caller, he or she generally chooses to hang up the voice call and use other chat software to reply by text, or apply to go out to answer the voice call, or lower the call volume and answer the call directly.

[0004] However, in the above method, the receiver cannot respond to the caller's message in real time after hanging up the voice call, that is, it takes a while for the receiver to reply to the caller after hanging up the voice call, resulting in the receiver being unable to respond to the caller's message in real time. As a result, the real-time and smoothness of voice calls on electronic devices are poor. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a call method, device, electronic device and readable storage medium, which can improve the real-time and fluency of electronic device calls.

[0006] In a first aspect, an embodiment of the present application provides a calling method, which is applied to a first electronic device, and the method includes: when the first electronic device is talking with a second electronic device, receiving a first voice message sent by the second electronic device; converting the first voice message into a first text message, and displaying the first text message; obtaining second text message input by a user; converting the second text message into a second voice message based on a first timbre, and sending the second voice message to the second electronic device.

[0007] In a second aspect, an embodiment of the present application provides a calling device, which includes: a receiving module, a processing module, a display module and a sending module. The receiving module is used to receive a first voice message sent by the second electronic device when the first electronic device is talking with the second electronic device. The processing module is used to convert the first voice message received by the receiving module into a first text message. The display module is used to display the first text message obtained by the processing module. The receiving module is also used to obtain second text information input by a user. The processing module is also used to convert the second text information obtained by the receiving module into a second voice message based on the first timbre. The sending module is used to send the second voice message obtained by the processing module to the second electronic device.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, when a first electronic device is talking to a second electronic device, the first voice information from the second electronic device can be converted into a first text information, and the first text information can be displayed; then the second text information input by the user can be converted into a second voice information based on the first timbre, and the second voice information can be sent to the second electronic device. In this solution, after receiving the voice of the user of the second electronic device, the first electronic device can display the voice to the user of the first electronic device in the form of text, so that the user of the first electronic device can understand the content of the call of the user of the second electronic device, so that the user of the first electronic device can directly reply to the text according to the displayed text information, and then the first electronic device can convey the content of the reply to the user of the second electronic device in the form of voice with the set first timbre, so that the user of the second electronic device can understand the content of the call of the user of the first electronic device. In this way, through this voice-text-voice call mode, even if the environment where the user is located is not convenient for voice calls, it can respond to the content of the call of the user of the second electronic device in real time, improve the real-time and fluency of the electronic device in voice calls, and thus improve the interactive experience of the user during the call. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is one of the flow charts of the calling method provided in the embodiment of the present application;

[0014] Figure 2 It is a schematic diagram of the process of converting speech to text provided in an embodiment of the present application;

[0015] Figure 3 It is a flowchart of converting text to speech provided by an embodiment of the present application;

[0016] Figure 4 is a schematic diagram of a timbre and language type setting interface provided in an embodiment of the present application;

[0017] Figure 5 This is the second flowchart of the calling method provided in the embodiment of the present application;

[0018] Figure 6 is a schematic diagram of a first interface provided in an embodiment of the present application;

[0019] Figure 7 This is the third flowchart of the calling method provided in the embodiment of the present application;

[0020] Figure 8 This is the fourth flowchart of the calling method provided in the embodiment of the present application;

[0021] Fig. 9This is the fifth flowchart of the calling method provided in the embodiment of the present application;

[0022] Fig.10 This is the sixth flowchart of the calling method provided in the embodiment of the present application;

[0023] Fig.11 is a schematic diagram of a call request interface provided in an embodiment of the present application;

[0024] Fig.12 This is the seventh flowchart of the calling method provided in the embodiment of the present application;

[0025] Fig.13 is a schematic diagram of a communication device provided in an embodiment of the present application;

[0026] Fig.14 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Fig.15 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The terms "at least one (item)", "at least one of" and the like in this application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0031] The following describes in detail the calling method provided by the embodiment of the present application through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0032] The embodiments of the present application can be applied to a scenario where a user makes a call through an electronic device, such as a telephone call scenario, a video call scenario, or a voice call scenario. Exemplarily, the above-mentioned call scenarios can be a situation where the user is currently in an immersive scene and the environment in which the user is located is not convenient for voice calls.

[0033] The existing calling method adopts a voice-to-voice calling method. For example, when user A and user B are talking, user A's electronic device (hereinafter referred to as electronic device A) needs to receive the voice from user B's electronic device (hereinafter referred to as electronic device B) and transmit it to user A, and also needs to receive the voice input by user A and send it to electronic device B. If user A is currently in an immersive scene, it is not convenient for him to receive the voice from electronic device B or input the voice to electronic device A. Therefore, this calling method cannot realize the real-time voice call between user A and user B.

[0034] For example, the immersive scene currently in which user A is located is a conference room. When user A and user B need to have a real-time voice call, since the current immersive scene is not convenient for users to have a voice call, user A can choose to hang up directly and use other chat software to reply by text, or choose to apply to go out for a voice call or lower the call volume to answer the call directly.

[0035] However, in the above method, if user A chooses to hang up directly, he cannot respond to the message sent by user B in real time through the voice call, especially the emergency message sent by user B in real time; if user A chooses to use other chat software for text reply, then if user B is a special user, such as the elderly and children who cannot use the chat software, he cannot respond to the emergency message sent by user B in real time; if user A chooses to apply to go out for a voice call, it will interrupt the normal progress of the meeting and affect other participants; if user A chooses to lower the call volume and answer the call directly, it will interfere with other participants and reduce the quality of the voice call. In this way, the real-time and fluency of voice calls on electronic devices are poor.

[0036] The embodiment of the present application provides a call method. When a first electronic device and a second electronic device are talking, the first voice information from the second electronic device can be converted into a first text information and the first text information can be displayed; then the second text information input by the user can be converted into a second voice information based on the first timbre, and the second voice information can be sent to the second electronic device. In this solution, after receiving the voice of the user of the second electronic device, the first electronic device can display the voice to the user of the first electronic device in the form of text, so that the user of the first electronic device can understand the content of the call of the user of the second electronic device, so that the user of the first electronic device can directly reply to the text according to the displayed text information, and then the first electronic device can convey the content of the reply to the user of the second electronic device in the form of voice with the set first timbre, so that the user of the second electronic device can understand the content of the call of the user of the first electronic device. In this way, through this voice-text-voice call mode, even if the environment where the user is located is not convenient for voice calls, it can respond to the content of the call of the user of the second electronic device in real time, improve the real-time and fluency of the electronic device in voice calls, and thus improve the interactive experience of the user during the call.

[0037] The execution subject of the call method provided in the embodiment of the present application may be a call device, and the call device may be an electronic device, or a functional module or functional entity in an electronic device. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0038] Figure 1 FIG. 1 is a flow chart of a calling method provided by an embodiment of the present application. Figure 1 As shown, the calling method provided in the embodiment of the present application may include the following steps 201 to 204.

[0039] Step 201: When a first electronic device is talking to a second electronic device, the first electronic device receives a first voice message sent by the second electronic device.

[0040] In the embodiment of the present application, the conversation between the first electronic device and the second electronic device can be any conversation method using voice as the medium for real-time communication, for example, making a mobile phone call, a voice call in certain applications, a video call, etc.

[0041] For example, taking a conversation between a first electronic device and a second electronic device by dialing a mobile phone as an example, a dedicated circuit channel is established between the first electronic device and the second electronic device, and voice transmission is performed over a mobile communication network.

[0042] For example, taking a call between a first electronic device and a second electronic device through a voice call function in certain applications, the first electronic device and the second electronic device are both connected to the network and rely on the Internet to transmit voice data.

[0043] In the embodiment of the present application, the first electronic device may receive the first voice information through a microphone module of the first electronic device.

[0044] In the embodiment of the present application, the first electronic device receiving the first voice information sent by the second electronic device may include: the first electronic device receiving the voice content of the first voice information, where the voice content refers to the specific content that the user wants to express through voice.

[0045] Optionally, in the embodiment of the present application, the first electronic device may also receive at least one of the timbre of the user's voice, the language type, etc. corresponding to the first voice information. The timbre refers to the characteristic performance of the user's voice, and the language type refers to the type of language used in the voice content of the first voice information, for example, Mandarin, Shaanxi dialect, Cantonese, etc.

[0046] Optionally, in an embodiment of the present application, the first electronic device may input the first voice information into a language type model, and perform language type recognition on the first voice information through the language type model to obtain the above-mentioned language type.

[0047] Optionally, in an embodiment of the present application, the first electronic device may train the above-mentioned language type model based on the constructed language type database. The language type database may be a language type database expanded using data enhancement technology, and may include multiple language types, such as Mandarin, Shaanxi dialect, Cantonese, etc. Specifically, the first electronic device may input the language type database into the initial language type model, optimize the initial language type model by adopting a transfer learning method, and obtain a trained language type model, thereby improving the accuracy of language type model recognition.

[0048] Alternatively, if Figure 2 As shown, in an embodiment of the present application, a first electronic device can utilize the above-mentioned language type model to receive voice feature information of a first voice information, where the voice feature information may include features such as spectrum, tone, and resonance peak, and then convert the voice feature information into a voice feature vector, and match the voice feature vector with all language types in the above-mentioned language type model, thereby determining the matched language type as the language type of the above-mentioned first voice information.

[0049] It should be noted that the matching of the above-mentioned speech feature vector and a language type means that the similarity between the speech feature vector and the language type is greater than or equal to a threshold (eg, 98%), or the similarity between the speech feature vector and the language type is the greatest.

[0050] Alternatively, if Figure 2 As shown, in an embodiment of the present application, after receiving the first voice information, the first electronic device can first predict the language type range corresponding to the first voice information based on the address information of the second electronic device, and then when recognizing the first voice information through the above-mentioned language type model, the above-mentioned voice feature vector can be preferentially matched with the language type corresponding to the language type range in the above-mentioned language type model, thereby determining the matched language type as the language type of the above-mentioned first voice information. In this way, the scope of language type recognition can be narrowed by this method, and the efficiency of electronic devices in language type recognition can be further accelerated.

[0051] It should be noted that the first electronic device may also use other methods that can accurately identify the language type to identify the language type, and the embodiments of the present application are not limited to this.

[0052] Step 202: The first electronic device converts the first voice information into first text information and displays the first text information.

[0053] Optionally, in an embodiment of the present application, the first text information may include text corresponding to all voice contents of the first voice information, or may include text corresponding to key voice contents of the first voice information.

[0054] Optionally, in the embodiment of the present application, the first text information may include at least one of Chinese character information and English information. In addition, the first text information may also include symbol information (such as punctuation marks or emoticons, etc.).

[0055] Optionally, in the embodiment of the present application, Figure 2As shown, the first electronic device can use ASR (Automatic Speech Recognition) technology to convert the above-mentioned first voice information into the above-mentioned first text information. Among them, ASR technology is a technology that converts human voice into input data readable by electronic devices. ASR technology is implemented through three main parts: signal processing, speech recognition and result processing. Among them, signal processing is to process the original audio signal into a form that can be used for speech recognition, such as noise removal and speech enhancement; speech recognition is to convert the processed signal into text, commands or operating instructions; result processing is to further process and apply the recognition results. Since the language types of different voices may be different, in the process of using ASR technology to convert voice into text information, a large language type model can be used. The large language type model is a model trained based on a large-scale corpus, covering text information corresponding to voices of various language types.

[0056] For example, taking the language type of the first voice information as Cantonese, in the process of the first electronic device converting the first voice information into the first text information, the ASR technology and the Cantonese language type large model are combined to accurately identify the specific content of the first voice information in Cantonese and output the corresponding first text information. This method can accurately convert voice into corresponding text information for different language types of voices, thereby improving the flexibility of the first electronic device in converting the voice into text information.

[0057] Optionally, in an embodiment of the present application, after the first electronic device identifies the language type of the first voice information, it can associate the language type with the second electronic device and store the associated information. Thus, when the first electronic device and the second electronic device talk again, the first electronic device can make a matching judgment between the language type of the voice from the second electronic device this time (hereinafter referred to as the current language type) and the language type in the above associated information. If the current language type is consistent with the language type in the above associated information, the first electronic device continues to use the previous language type model to convert the voice from the second electronic device this time into corresponding text information; if the current language type is inconsistent with the language type in the above associated information, the first electronic device can use the language type model corresponding to the current language type to convert the voice from the second electronic device this time into corresponding text information.

[0058] It should be noted that the first electronic device may also use other methods that can accurately convert voice into text information to convert voice into text information, and the embodiment of the present application is not limited to this.

[0059] In an embodiment of the present application, the first electronic device may use streaming input technology to display the above-mentioned first text information. Streaming input refers to data being input into an electronic device in the form of a continuous data stream for processing. This processing method does not require all data to be loaded into the memory at once, but rather processes while reading, which is suitable for processing large amounts of data or real-time data processing scenarios. Specifically, the first electronic device is capable of processing a continuous voice stream, and converting the newly added voice into text information in real time during the input of the first voice information, without having to wait for the entire first voice information input to be completed before converting. This method not only improves the efficiency of the first electronic device in converting the first voice information into the first text information, but also enables the first electronic device to interact more naturally with the user to whom the first electronic device belongs.

[0060] Step 203: The first electronic device obtains second text information input by the user.

[0061] In the embodiment of the present application, the second text information may be the content of a call communicated from the user of the first electronic device to the user of the second electronic device.

[0062] Optionally, in the embodiment of the present application, the second text information may include at least one of Chinese character information and English information. In addition, the second text information may also include symbol information (such as punctuation marks or emoticons, etc.).

[0063] Optionally, in the embodiment of the present application, the manner in which the first electronic device obtains the second text information may include but is not limited to at least one of the following: the first electronic device obtains the second text information input by the user through the keyboard; the first electronic device obtains the second text information selected by the user through the quick reply key; the first electronic device obtains the second text information input by the user by handwriting; the first electronic device obtains the second text information obtained by the user through copying and pasting; the first electronic device obtains the scanned second text information, etc. The specific method may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0064] Optionally, in the embodiment of the present application, the second text message may be a message actively sent by the user of the first electronic device to the user of the second electronic device, or may be a reply message made by the user of the first electronic device to the user of the second electronic device based on the first text message, etc. The specific information may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0065] Step 204: The first electronic device converts the second text information into second voice information based on the first timbre, and sends the second voice information to the second electronic device.

[0066] Optionally, in an embodiment of the present application, the first electronic device may use TTS (Text To Speech) technology to convert the second text information into the second voice information. TTS technology is a type of speech synthesis application, which converts files stored in electronic devices, such as help files or web pages, into natural speech output. TTS technology can not only help people with visual impairments read information on electronic devices, but also increase the readability of text documents. TTS technology is used in voice-driven emails and sound-sensitive systems, and is often used in conjunction with voice recognition programs.

[0067] In the embodiment of the present application, the second voice information is the voice information corresponding to the second text information.

[0068] In the embodiment of the present application, the second voice information may include: voice content, where the voice content refers to the specific content to be expressed by the second text information.

[0069] Optionally, in the embodiment of the present application, the voice content of the second voice information may include the voice corresponding to the entire text content of the second text information, or may include the voice corresponding to the key text content of the second text information.

[0070] Optionally, in the embodiment of the present application, the second voice information may further include at least one of: sound timbre, language content, etc. The sound timbre refers to the characteristic performance of the sound corresponding to the second voice information, and the language type refers to the type of language used in the voice content of the second voice information, such as Mandarin, Shaanxi dialect, Cantonese, etc.

[0071] In the embodiment of the present application, the first electronic device can convert the second text information into second voice information based on the first timbre, and send the second voice information to the second electronic device.

[0072] Optionally, in the embodiment of the present application, the first electronic device may use TTS technology in combination with the timbre parameters of the first timbre to convert the second text information into second voice information that conforms to the first timbre, and the timbre parameters may include pitch, loudness, sound quality, etc. In this way, the flexibility of the first electronic device in converting the second text information into the second voice information is improved.

[0073] Optionally, in the embodiment of the present application, the above-mentioned first timbre may include any one of the following: a system default timbre, a user-set timbre, etc.

[0074] Optionally, in an embodiment of the present application, a user can pre-set the timbre of the text information when it is converted to voice during a call in the voice setting interface, and the voice setting interface includes timbre setting options, such as a recording audio option and an uploading audio option. The recording audio option is used for the user to record audio, and perform timbre recognition on the audio recorded by the user to set the timbre of the recorded audio as the timbre when the text information is converted to voice during a call. The uploading audio option is used for the user to upload historical audio, and perform timbre recognition on the historical audio to set the timbre of the historical audio as the timbre when the text information is converted to voice during a call. Thus, the user can input any option in the recording audio option and the uploading audio option to set the timbre of the voice (for example, the first timbre), so that when the text information is subsequently converted to voice during a call, the timbre pre-set by the user can be directly used to convert the text information into the corresponding voice.

[0075] Optionally, in the embodiment of the present application, the first electronic device may convert the second text information into second voice information using the first language type, and send the second voice information to the second electronic device.

[0076] In the embodiment of the present application, the first language type may include any one of the following: a language type determined based on the language type of the first voice information, a system default language type, and a language type set by the user.

[0077] Alternatively, if Figure 3 As shown, in the embodiment of the present application, the first electronic device can use TTS technology combined with a large language type model to convert the second text information into a second voice information that conforms to the first language type. The large language type model is a model trained based on a large-scale corpus, covering text information corresponding to voices of various language types. In this way, the flexibility of the first electronic device in converting the second text information into the second voice information is improved.

[0078] Exemplarily, taking the first language type as a language type determined based on the language type of the first voice information as an example, for example, the first electronic device recognizes that the language type of the first voice information is Sichuan dialect. In the process of the first electronic device converting the second text information into the second voice information, the first electronic device combines the TTS technology and the Sichuan dialect model to accurately convert the second text information into the corresponding second voice information in Sichuan dialect, and the first electronic device sends the second voice information to the second electronic device.

[0079] Optionally, in an embodiment of the present application, the first electronic device can convert the second text information into the second voice information by streaming input. Specifically, the first electronic device can process a continuous text stream, and convert the newly added text into voice information in real time during the input of the second text information, and splice the voice fragment information into a coherent voice data stream, without having to wait for the entire second text information to be input before converting. This method not only improves the efficiency of the first electronic device in converting the second text information into the second voice information, but also enables the first electronic device to interact more naturally with the user of the first electronic device.

[0080] Optionally, in the embodiment of the present application, the first electronic device may use the language type of the first voice information as the first language type.

[0081] Optionally, in the embodiment of the present application, the default language type of the above system may be Mandarin.

[0082] For example, taking the first language type as the system default language type, in the process of the first electronic device converting the second text information into the second voice information, the combination of TTS technology and the Mandarin large model can accurately convert the second text information into the corresponding Mandarin second voice information, and the first electronic device sends the second voice information to the second electronic device.

[0083] Optionally, in an embodiment of the present application, the user pre-sets the language type when converting text information into voice during a call in the voice setting interface. The voice setting interface includes language type setting options, such as at least one language type option. The user can input any of these options to set the language type of the voice, so that when the text information is subsequently converted into voice during the call, the language type pre-selected by the user can be used to directly convert the text information into the corresponding voice.

[0084] For example, the electronic device is a mobile phone. Figure 4 As shown, the mobile phone displays a voice setting interface 20, which includes timbre setting options and language type setting options, namely, recording audio option 21 and uploading audio option 22, and at least one language type option 23. The user can record the audio in real time by clicking the recording audio option 21, or can upload the historical audio by clicking the uploading audio option 22, so that the mobile phone can perform timbre recognition on the recorded audio or the uploaded historical audio, and set the recognized timbre as the timbre when the text information is converted into voice during the call. The user can select any language type option in at least one language type option 23, so that the mobile phone sets the language type corresponding to the language type option selected by the user as the language type when the text information is converted into voice during the call.

[0085] In this way, the first electronic device can convert the second text information input by the user of the first electronic device into a second voice information based on the first timbre, and send the second voice information to the second electronic device, and the user of the second electronic device can receive the message content conveyed by the user of the first electronic device in the form of voice, thereby realizing a real-time voice call between the user of the first electronic device and the user of the second electronic device, and expanding the scenario of the first electronic device making a voice call. Moreover, the second electronic device is unaware of the series of conversion processes of voice-text-voice conversion, and the user of the first electronic device can make a real-time and smooth voice call without considering the identity of the user of the second electronic device and the current occasion, thereby improving the convenience, real-time and smoothness of the electronic device making a voice call, and expanding the possibility of the electronic device making a voice call.

[0086] The embodiment of the present application provides a call method. When a first electronic device and a second electronic device are talking, the first voice information from the second electronic device can be converted into a first text information and the first text information can be displayed; then the second text information input by the user can be converted into a second voice information based on the first timbre, and the second voice information can be sent to the second electronic device. In this solution, after receiving the voice of the user of the second electronic device, the first electronic device can display the voice to the user of the first electronic device in the form of text, so that the user of the first electronic device can understand the content of the call of the user of the second electronic device, so that the user of the first electronic device can directly reply to the text according to the displayed text information, and then the first electronic device can convey the content of the reply to the user of the second electronic device in the form of voice with the set first timbre, so that the user of the second electronic device can understand the content of the call of the user of the first electronic device. In this way, through this voice-text-voice call mode, even if the environment where the user is located is not convenient for voice calls, it can respond to the content of the call of the user of the second electronic device in real time, improve the real-time and fluency of the electronic device in voice calls, and thus improve the interactive experience of the user during the call.

[0087] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 5 As shown, the above step 202 can be specifically implemented through the following steps 202a and 202b, and the above step 203 can be specifically implemented through the following steps 203a to 203c.

[0088] Step 202a: The first electronic device converts the first voice information into a first text information and displays a first interface.

[0089] In the embodiment of the present application, the first interface includes a text display area and a text input area.

[0090] In the embodiment of the present application, the text display area of ​​the first interface can be used to display text information, which can include text information obtained by voice conversion (such as the first text information) and text information input by the user (such as the second text information). In this way, the user can obtain the text information of the caller (such as the second electronic device) and the content of the message he / she replied to the caller from the text display area.

[0091] In the embodiment of the present application, the text input area of ​​the above-mentioned first interface can be used for the user of the first electronic device to perform text input operations.

[0092] Optionally, in the embodiment of the present application, the above text input operation includes but is not limited to: performing text input operation through keyboard input; performing text input operation by selecting a quick reply key; performing text input operation through handwriting input; performing text input operation through copying and pasting; performing text input operation by scanning image content or document content, etc. The specific operation can be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0093] Optionally, in the embodiment of the present application, when the first electronic device displays any interface, if the first electronic device receives a call request (i.e., an incoming call) from the second electronic device, the first electronic device may update the arbitrary interface to the above-mentioned first interface, or may also display a window in the arbitrary interface, the window including the above-mentioned first interface. The arbitrary interface may be a lock screen interface, a desktop or an application interface, etc.

[0094] For example, Figure 6 As shown, the mobile phone displays a first interface 60, which includes a text display area 62 and a text input area 63. The text display area 62 is used to display text information, and the text input area 63 is used for the user to perform text input operations.

[0095] Optionally, in an embodiment of the present application, the size of the text display area can be adaptively adjusted according to the amount of text information displayed, and the size of the text input area can be adaptively adjusted according to the amount of text information input by the user.

[0096] Step 202b: The first electronic device displays the first text information in the text display area.

[0097] In the embodiment of the present application, the first electronic device can display the first text message in the above-mentioned text display area, so that the user of the first electronic device can understand the message content conveyed by the user of the second electronic device more real-time and intuitively through text in an immersive scene. Moreover, this method is highly intuitive, and enables the user of the first electronic device to make a corresponding reply according to the content of the first text message, respond to the message content in a timely manner, and improve the real-time and fluency of the electronic device in voice calls.

[0098] Optionally, in the embodiment of the present application, when the first electronic device displays the first text information in the text display area, it may also display an identifier corresponding to the second electronic device, such as a user avatar of a user to which the second electronic device belongs.

[0099] Step 203a: The first electronic device receives a first input of the second text information by the user in the text input area.

[0100] In the embodiment of the present application, the first input is used to trigger the first electronic device to send the second text message to the second electronic device.

[0101] Optionally, in an embodiment of the present application, the first input may include a user input of the content of the second text information and an input of a sending control, where the sending control is used to trigger the sending of the text information input by the user.

[0102] Step 203b: The first electronic device displays second text information in the text display area in response to the first input.

[0103] Optionally, in the embodiment of the present application, when the first electronic device displays the second text information in the text display area, it may also display an identifier corresponding to the first electronic device, such as a user avatar of a user to which the first electronic device belongs.

[0104] For example, Figure 6 As shown, the mobile phone displays a first interface 60, which includes a text display area 62 and a text input area 63; after the mobile phone receives a first voice message and converts it into a first text message, such as "Did you watch yesterday's badminton match?", the mobile phone can display the first text message in the text display area 62, and after the user enters a second text message in the text input area 63, the mobile phone can display the second text message in the text display area 62, such as "I saw it", so that the user can intuitively view the message content of the caller and the message content of his reply in the text display area 62.

[0105] Step 203c: The first electronic device obtains second text information from the text display area.

[0106] In the embodiment of the present application, the second text information in the above-mentioned text display area is the information finally determined to be sent to the second electronic device. The first electronic device can obtain the second text information from the text display area and send the second text information to the second electronic device in the form of voice.

[0107] In this way, when the first electronic device is talking to the second electronic device, the first electronic device displays the first interface, and the first electronic device converts the first voice information from the second electronic device into a first text information, and displays the first text information on the first interface; then, based on the first timbre, the second text information input by the user displayed in the text display area can be converted into a second voice information, and the second voice information is sent to the second electronic device. In this solution, the first electronic device displays the first interface, and the text display area in the first interface can display the text content of the conversation between the user of the first electronic device and the user of the second electronic device in real time and intuitively, and enables the user of the first electronic device to make a corresponding reply according to the content of the first text information, thereby improving the real-time and fluency of the voice call of the electronic device.

[0108] Optionally, in the embodiment of the present application, the first interface may further include a text pending area. Figure 5 ,like Figure 7 As shown, before "displaying the second text message in the text display area" in the above step 203b, the calling method provided in the embodiment of the present application also includes the following step 301, and the "displaying the second text message in the text display area" in the above step 203b can be specifically implemented by the following step 203b1.

[0109] Step 301: A first electronic device displays second text information in a text pending area in response to a first input.

[0110] Step 203b1: when the correlation between the second text information in the text pending area and the first text information is greater than or equal to a first threshold, the first electronic device updates the second text information from the text pending area to the text display area for display.

[0111] In an embodiment of the present application, after the user makes a first input for the above-mentioned second text information in the text input area, the first electronic device can first display the second text information in the text pending area to confirm whether the second text information is the message content of the user's reply to the above-mentioned first voice information.

[0112] Optionally, in the embodiments of the present application, the first electronic device may slice the second text information in the text to-be-determined area to obtain at least one key information, so as to compare the relevance between the at least one key information and the first text information. Among them, the key information here may include, but is not limited to, at least one of the following: words, phrases, sentences, etc.

[0113] Optionally, in the embodiments of the present application, the first electronic device may perform semantic relevance analysis on the at least one key information and the first text information to obtain the relevance between the second text information and the first text information.

[0114] It can be understood that semantic relevance is an important concept in the field of information science and technology. It involves the relevance and correlation between texts, data, or information, and is particularly applied to natural language processing and machine learning. Semantic relevance not only involves the literal meaning of the text, but also includes the understanding of the intention, context, and context behind the text. In natural language processing, semantic relevance models are used to evaluate the relevance or correlation between two text segments. The greater the relevance between the second text information in the text to-be-determined area and the first text information, the closer the two are in meaning or content.

[0115] Optionally, in the embodiments of the present application, the first threshold includes, but is not limited to, any one of the following: the threshold default in the first electronic device system, the specific threshold set by the user to which the first electronic device belongs. Specifically, it can be determined according to actual usage requirements, and the embodiments of the present application do not limit this. For example, the first threshold may be 90% or 95%.

[0116] Optionally, in the embodiments of the present application, when the relevance between the second text information in the text to-be-determined area and the first text information is greater than or equal to the first threshold, the first electronic device may first correct and proofread the grammar of the second text information in the text to-be-determined area, and then update the second text information after grammar correction and proofreading to the text display area for display.

[0117] Exemplarily, the text information in the text display area is "The weather is nice today", and the text information in the text to-be-determined area is "Listening to the wind and kylin". Slicing the text information in the text to-be-determined area, the result is ["Listening", "to the wind and", "kylin", "of"], and after semantic relevance analysis, the relevance between the text information in the text to-be-determined area and the text information in the text display area is greater than the first threshold. Correcting and proofreading the grammar of the text information in the text to-be-determined area, the grammar correction and proofreading result is ["Quite", "windy and sunny", "of"], and updating "Quite windy and sunny" to the text display area.

[0118] In the embodiment of the present application, the correlation between the second text information in the text pending area and the first text information is greater than or equal to the first threshold value, indicating that the second text information is close to the first text information in meaning or content, and the second text information in the text pending area can be used as a reasonable reply from the user of the first electronic device to the message sent by the user of the second electronic device, and the first electronic device performs grammatical correction and proofreading on the second text information in the text pending area before updating the second text information from the text pending area to the text display area for display. In the above manner, semantic correlation analysis, grammatical correction and proofreading are used to improve the coherence, accuracy and fluency of the voice call of the first electronic device.

[0119] Optionally, in an embodiment of the present application, when the first electronic device is updating the second text information from the text pending area to the text display area for display, the first electronic device may remove the second text information from the text pending area to ensure that the content of the second text information in the text display area will not be repeated, thereby improving the fluency of the voice call of the first electronic device.

[0120] It should be noted that the display method of the first text information and the second text information in the above-mentioned text display area can be arbitrary, as long as it can enable the user of the first electronic device to distinguish the content and order of the first text information and the second text information. The specific method can be determined according to actual usage requirements, and the embodiment of the present application does not limit this.

[0121] In this way, by confirming the correlation between the second text information in the text pending area and the first text information, when the correlation is greater than or equal to the first threshold, the first electronic device updates the second text information from the text pending area to the text display area for display, thereby ensuring that the text content displayed in the text display area is close in meaning or content to the message content conveyed by the user of the second electronic device, conforms to the logic of the voice call, and improves the continuity, accuracy and fluency of the voice call conducted by the first electronic device.

[0122] Optionally, in the embodiment of the present application, after the above step 301, the calling method provided in the embodiment of the present application further includes the following step 302.

[0123] Step 302: When the correlation between the second text information in the text pending area and the first text information is less than a first threshold, if within a first time period, the first electronic device detects that text information is added to the text pending area, it is determined whether the correlation between the updated text information in the text pending area and the first text information is greater than or equal to the first threshold.

[0124] In an embodiment of the present application, the correlation between the second text information in the text pending area and the first text information is less than the first threshold value, indicating that the second text information is not relevant to the first text information in meaning or content, and the second text information in the text pending area is not suitable as a reply from the user of the first electronic device to the message sent by the user of the second electronic device, and it is also necessary to determine whether the user of the first electronic device has made corresponding supplements to the second text information in the text pending area within the first time period.

[0125] In an embodiment of the present application, if within a first time period, the first electronic device detects that text information is added to the text pending area, indicating that the user to which the first electronic device belongs has made corresponding supplements to the second text information in the text pending area, then the previously determined correlation between the second text information and the first text information is invalid. At this time, the updated text information in the text pending area may have a certain semantic correlation with the first text information. Therefore, in order to improve the accuracy of voice calls by the first electronic device, it is necessary to determine whether the correlation between the updated text information in the text pending area (that is, including the second text information and the newly added text information) and the first text information is greater than or equal to the first threshold.

[0126] When the correlation between the updated text information in the text pending area and the first text information is greater than or equal to the first threshold, the first electronic device updates all text information in the text pending area from the text pending area to the text display area for display. When the correlation between the updated text information in the text pending area and the first text information is less than the first threshold, step 302 is repeated.

[0127] Optionally, in an embodiment of the present application, if the first electronic device does not detect that text information is added to the text pending area within the first time period, the first electronic device may update the second text information from the text pending area to the text display area for display.

[0128] In an embodiment of the present application, if the first electronic device does not detect the addition of text information in the text pending area within the first time period, it means that the user of the first electronic device has not supplemented the second text information in the text pending area. In order to reduce the waiting time of the user of the second electronic device, the first electronic device directly updates the second text information from the text pending area to the text display area for display, so that the first electronic device obtains the second text information of the user's reply from the text display area and sends it to the second electronic device in the form of voice.

[0129] Optionally, in the embodiment of the present application, the first duration may include but is not limited to any of the following: a default duration of the first electronic device system, a specific duration set by the user of the first electronic device. The specific duration may be determined according to actual usage requirements, and the embodiment of the present application does not limit this. For example, the first duration may be 3 seconds.

[0130] Exemplarily, the text information in the text display area is "What's wrong with you", and the text information in the text pending area is "Yes". At this time, because the text information in the text pending area only has one word, text segmentation is skipped and semantic correlation analysis is performed directly. When the correlation is relatively low, it is necessary to check whether there is new text information in the text pending area within the first time length. If there is no new text information, "Yes" is directly updated to the text display area. If there is new text information, the newly added text information is spliced ​​after "Yes". For example, if the new text information is "something", "something" is re-sliced, and the "something" obtained after text slicing is subjected to semantic correlation analysis with "What's wrong with you".

[0131] For example, Figure 6 As shown, the mobile phone displays a first interface 60, which also includes a text pending area 64. After the user makes a first input of the second text information in the above-mentioned text input area 63, the mobile phone can display the second text information in the text pending area 64, and then the mobile phone can compare the relevance of the second text information in the text pending area 64 with the first text information in the above-mentioned text display area 62, so as to update the second text information in the text pending area 64 to be displayed in the above-mentioned text display area 62 when the relevance between the two is greater than or equal to the first threshold.

[0132] It should be noted that the execution relationship between the above step 302 and the above step 203b1 is an "or" relationship. That is, when the above step 203b1 is executed, the above step 302 is not executed; or, when the above step 302 is executed, the above step 203b1 is not executed.

[0133] Optionally, in the embodiment of the present application, combined with Figure 5 ,like Figure 8 As shown, the "displaying the first interface" in the above step 202a can be specifically implemented through the following step 202a1.

[0134] Step 202a1: The first electronic device converts the first voice information into the first text information, and when the first electronic device is interconnected with the third electronic device, shares the first interface with the third electronic device for display.

[0135] Optionally, in the embodiment of the present application, the interconnection between the first electronic device and the third electronic device may include but is not limited to any of the following: Bluetooth connection, wireless network connection, USB connection, etc. The specific connection may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0136] It is understandable that Bluetooth is a short-range wireless communication technology suitable for transmitting data between different electronic devices. By turning on the Bluetooth function of two electronic devices and pairing them, data transmission and sharing can be achieved. Wireless network connections mainly include Wi-Fi hotspot connections and mobile network sharing (such as USB network sharing). Through Wi-Fi hotspot connections, one electronic device can be used as a hotspot, and another electronic device can be connected to the hotspot via Wi-Fi to achieve network sharing; mobile network sharing is usually connected via a USB cable to share the mobile network of one device with another device. USB connection is a common physical connection method that directly connects two electronic devices via a USB cable.

[0137] For example, taking the case where the first electronic device and the third electronic device are interconnected via a Bluetooth connection, by turning on the Bluetooth functions of the first electronic device and the third electronic device and pairing them, the first electronic device shares the first interface with the third electronic device.

[0138] For example, taking the example of a first electronic device and a third electronic device being interconnected via a wireless network connection, the first electronic device is used as a hotspot, the third electronic device is connected to the hotspot of the first electronic device via Wi-Fi, and the first electronic device shares the first interface with the third electronic device.

[0139] For example, taking the example of the first electronic device and the third electronic device being interconnected via a USB connection, the first electronic device and the third electronic device are directly connected via a USB cable, and the first electronic device shares the first interface with the third electronic device.

[0140] In this way, the first electronic device can share the first interface to the third electronic device for display. By using this device to synchronously display the message content conveyed by the user, real-time response to calls between electronic devices can be achieved without affecting the use of the first electronic device, thereby improving the flexibility and scalability of electronic devices for voice calls.

[0141] Optionally, in the embodiment of the present application, combined with Figure 5 ,like Fig. 9 As shown, the above step 204 can be specifically implemented through the following steps 204a to 204d.

[0142] Step 204a: The first electronic device receives a third input from the user to the tone selection control of the first interface.

[0143] In the embodiment of the present application, the third input is used to trigger the first electronic device to display a tone selection interface.

[0144] Optionally, in the embodiment of the present application, the third input includes but is not limited to: the user of the first electronic device performs touch input on the tone selection control of the first interface through a touch device such as a finger or a stylus, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0145] Step 204b: The first electronic device displays a tone selection interface in response to the third input.

[0146] In the embodiment of the present application, the above-mentioned tone selection interface may include at least one tone, and the user can select the required tone, such as the first tone, from the at least one tone.

[0147] Step 204c: The first electronic device receives the first timbre selected by the user in the timbre selection interface.

[0148] In an embodiment of the present application, after the first electronic device receives the third input from the user to the tone selection control of the first interface, the first electronic device displays the tone selection interface, so that the user can select the first tone used to convert the second text information into the second voice information through the tone selection interface.

[0149] Step 204d: The first electronic device converts the second text information based on the first timbre to obtain second voice information, and sends the second voice information to the second electronic device.

[0150] In an embodiment of the present application, the above-mentioned timbre selection interface includes at least one of an online timbre library, a preset timbre library, and a timbre library synthesized based on the voiceprint of at least one user and languages ​​of different language types.

[0151] Optionally, in the embodiment of the present application, the user can also select the required first timbre and first language type during the conversation between the first electronic device and the second electronic device, so as to convert the text information into corresponding voice based on the selected first timbre and first language type. For example, during the conversation between the first electronic device and the second electronic device, the first electronic device can display a first interface, which includes timbre and language type button controls for the user to select timbre and language type.

[0152] For example, Figure 6 As shown, the mobile phone displays a first interface 60, which includes a timbre and language type key control 61. The user can select and input the timbre and language type key control 61 to set the selected timbre and language type as the timbre and language type used when converting text information into speech.

[0153] Optionally, in the embodiment of the present application, combined with Figure 5 ,like Fig.10 As shown, before the above step 201, the calling method provided in the embodiment of the present application further includes the following step 200, and the above step 201 can be specifically implemented by the following step 201a.

[0154] Step 200: When receiving a call request from a second electronic device, the first electronic device displays a first control and a second control.

[0155] In the embodiment of the present application, the above-mentioned call request is used by the second electronic device to request to establish a call with the first electronic device, such as the telephone call, video call or voice call described in the above-mentioned embodiment.

[0156] In the embodiment of the present application, the first control is used to indicate the selection of a call mode for converting between voice and text, and the second control is used to indicate the selection of a voice call mode.

[0157] It should be noted that the voice-to-text conversion call mode can be understood as: during a call, when the first electronic device receives voice from other electronic devices, it can convert the voice into text, and when receiving text information input by the user of the first electronic device, it can convert the text information into voice to send the voice to other electronic devices. This call mode is imperceptible to the user of the second electronic device, that is, the user of the second electronic device sends voice and receives voice, and the first electronic device implements the voice-to-first text information and second text information-to-second voice information conversion process. Therefore, this call mode is called a voice-to-text conversion call mode, and can also be expressed as a voice-to-text-to-voice call mode.

[0158] Optionally, in an embodiment of the present application, upon receiving a call request from a second electronic device, the first electronic device may display a call request interface, which includes the first control and the second control. In addition, the call request interface may also include the user name, phone number, and location of the second electronic device.

[0159] Step 201a: When the first electronic device receives the second input to the first control, the first electronic device establishes a call connection between the first electronic device and the second electronic device, and receives the first voice information sent by the second electronic device.

[0160] In the embodiment of the present application, the second input is used to trigger the establishment of a call connection between the first electronic device and the second electronic device.

[0161] Optionally, in the embodiment of the present application, the second input includes but is not limited to: the user of the first electronic device performs touch input on the first control through a touch device such as a finger or a stylus, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0162] In an embodiment of the present application, after the first electronic device receives the second input of the user to the first control, the first electronic device can know that the call mode required by the user is a call mode in which voice and text are converted into each other. Then the first electronic device establishes a call connection with the second electronic device and receives the first voice message sent by the second electronic device. The first electronic device converts the first voice message into a first text message and displays the first interface, showing the first interface, i.e., the text input area and the text display area, to the user, so that the user can realize a call in which voice and text are converted into each other through the text input area and the text display area.

[0163] For example, Fig.11 As shown, when a mobile phone receives a call request from a second electronic device, the mobile phone can display a call request interface 10, wherein the call request interface 10 includes a first control and a second control, such as a text call control 11 and a voice call control 12. The text call control 11 is used to indicate the selection of a call mode for converting voice and text. The user can touch the text call control 11 to input so that the mobile phone establishes a call with the second electronic device, and updates the call request interface 10 to the above-mentioned Figure 6 A first interface 60 is shown.

[0164] Optionally, in the embodiment of the present application, after the first electronic device receives the fourth input of the user to the second control, the first electronic device can know that the call mode required by the user is a voice call mode, and the first electronic device can also convert the third voice input by the user of the first electronic device into a fourth voice that conforms to the first timbre based on the above-mentioned first timbre, and send the fourth voice to the second electronic device. In this way, when the user selects the voice call mode, the first electronic device can also independently select the first timbre, and output the voice of the user of the first electronic device to the user of the second electronic device with the first timbre, thereby improving the user's interactive experience during the call.

[0165] Optionally, in an embodiment of the present application, after the first electronic device successfully establishes a call with the second electronic device, the first electronic device may display a prompt message in the first interface, where the prompt message is used to prompt that the first electronic device has established a call connection with the second electronic device and is in a call state.

[0166] In this way, when the first electronic device receives a call request from the second electronic device, the first electronic device enters a call mode of converting between voice and text by responding to the second input confirmation of the first control, thereby providing a variety of call modes for the user of the first electronic device. When receiving a call request, the user of the first electronic device can select a suitable call mode according to actual conditions, thereby improving the flexibility of the first electronic device in making voice calls.

[0167] Optionally, in the embodiment of the present application, when the first electronic device receives the second input to the first control, the first electronic device may further send a prompt message to the second electronic device.

[0168] In the embodiment of the present application, the above prompt information is used to prompt the user of the second electronic device that the first electronic device adopts a call mode of converting between voice and text.

[0169] In an embodiment of the present application, when a call request is received from a second electronic device, the first electronic device displays a first control and a second control, and the user confirms that during the call, the first electronic device uses a call method of converting between voice and text through a second input to the first control. Then, the first electronic device can send a prompt message to the second electronic device to prompt the user of the second electronic device that the user of the first electronic device is communicating through a call method of converting between voice and text, so that the user of the second electronic device can reserve a certain amount of time for the user of the first electronic device during the call, thereby preventing the user of the second electronic device from mistakenly believing that the user of the first electronic device did not respond to the message conveyed by the user in a timely manner when the user does not know that the user of the first electronic device is communicating through a call method of converting between voice and text, thereby improving the effectiveness and flexibility of calls between electronic devices and improving human-computer interaction performance.

[0170] For example, Figure 6 As shown, the mobile phone displays a first interface 60, which includes a text display area 62. The text display area 62 includes prompt information, and the prompt information is used to prompt the user of the second electronic device that the first electronic device uses a call method that converts between voice and text. For example, the specific content of the prompt information may include: Hello, I am using text calls now, and the answer may be slow, please be patient.

[0171] The following is a schematic illustration of the implementation process of the call method provided in the embodiment of the present application through a specific implementation method. Fig.12 As shown, the implementation process of the calling method provided in the embodiment of the present application may include the following steps 71 to 81.

[0172] It should be noted that the implementation method here is illustrated by taking the first electronic device as mobile phone A, the user of mobile phone A as user A, the second electronic device as mobile phone B, the user of mobile phone B as user B, and the above call as a telephone call as an example.

[0173] Step 71: Mobile phone A receives an incoming call request from mobile phone B.

[0174] It should be noted that the above incoming call request can also be called a telephone call request, which is used by mobile phone B to request to establish a telephone call with mobile phone A.

[0175] Step 72: Mobile phone A displays a phone call request interface, which includes a text call control and a voice call control.

[0176] Step 73: User A may make a second input to the text call control.

[0177] Step 74: Mobile phone A establishes a telephone call with mobile phone B in response to the second input, and displays a first interface, which includes a text input area, a text pending area, and a text display area.

[0178] Step 75: Mobile phone A receives the voice message b from user B, converts the voice message b into text message b, and displays the text message b in the text display area.

[0179] Step 76: After user A views text information b in the text display area, user A makes a first input to text information a in the text input area.

[0180] Step 77: Mobile phone A displays text information a in the text pending area in response to the first input.

[0181] Step 78: Mobile phone A determines whether the correlation between all text information in the text pending area and text information b in the text display area is greater than or equal to a first threshold.

[0182] It should be noted that all text information in the above text pending area can be referred to as second text information.

[0183] It should be noted that, if yes, mobile phone A executes the following step 79 and step 80. If no, mobile phone A executes the following step 81.

[0184] Step 79: Mobile phone A displays the second text information in the text pending area in the text display area.

[0185] Step 80: Mobile phone A converts the second text information in the text display area into voice information a based on the first tone, and sends it to mobile phone B through a telephone call.

[0186] Step 81: Mobile phone A determines whether text information is added to the text pending area within the first time period.

[0187] It should be noted that, if yes, mobile phone A executes the above step 78. If no, mobile phone A executes the above steps 79 and 80.

[0188] It should be noted that for the relevant description of each step in this implementation mode, please refer to the description in the above embodiment, which will not be repeated here.

[0189] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application are not limited to this.

[0190] The communication method provided in the embodiment of the present application can be executed by a communication device. In the embodiment of the present application, the communication device provided in the embodiment of the present application is described by taking the communication device executing the communication method as an example.

[0191] Fig.13 A possible structural diagram of a communication device involved in some embodiments of the present application is shown. Fig.13 As shown, the calling device 30 may include: a receiving module 31, a processing module 32, a display module 33 and a sending module 34. The receiving module 31 is used to receive a first voice message sent by the second electronic device when the first electronic device is talking with the second electronic device. The processing module 32 is used to convert the first voice message received by the receiving module 31 into a first text message. The display module 33 is used to display the first text message obtained by the processing module 32. The receiving module 31 is also used to obtain the second text message input by the user. The processing module 32 is also used to convert the second text message obtained by the receiving module 31 into a second voice message based on the first timbre. The sending module 34 is used to send the second voice message obtained by the processing module 32 to the second electronic device.

[0192] In a possible implementation, the display module 33 is specifically used to display a first interface, which includes a text display area and a text input area; the first text information is displayed in the text display area. The receiving module 31 is used to receive a first input of a second text information from a user in the text input area. The display module 33 is also used to display the second text information in the text display area in response to the first input received by the receiving module 31. The receiving module 31 is specifically used to obtain the second text information from the text display area.

[0193] In a possible implementation, the first interface further includes a text pending area. The display module 33 is further configured to display the second text information in the text pending area before the second text information is displayed in the text display area. The display module 33 is specifically configured to update the second text information from the text pending area to the text display area for display when the correlation between the second text information in the text pending area and the first text information is greater than or equal to a first threshold.

[0194] In one possible implementation, the processing module 32 is also used to determine whether the correlation between the updated text information in the text pending area and the first text information is greater than or equal to the first threshold if, after the display module 33 displays the second text information in the text pending area, the correlation between the second text information in the text pending area and the first text information is less than the first threshold, if it is detected that text information is added to the text pending area within the first time length.

[0195] In a possible implementation, the display module 33 is specifically configured to share the first interface with the third electronic device for display when the first electronic device is interconnected with the third electronic device.

[0196] In a possible implementation, the receiving module 31 is further used to receive a third input from the user to the timbre selection control of the first interface. The display module 33 is further used to display the timbre selection interface in response to the third input received by the receiving module 31. The receiving module 31 is further used to receive a first timbre selected by the user in the timbre selection interface. The processing module 32 is specifically used to convert the second text information based on the first timbre to obtain the second voice information. The timbre selection interface includes at least one of the following: an online timbre library, a preset timbre library, and a timbre library synthesized based on the voiceprint of at least one user and languages ​​of different language types.

[0197] In a possible implementation, the display module 33 is also used to display a first control and a second control before the first electronic device establishes a call with the second electronic device, when a call request from the second electronic device is received. The first control is used to indicate the selection of a call mode for converting between voice and text, and the second control is used to indicate the selection of a voice call mode. The processing module 32 is also used to establish a call connection between the first electronic device and the second electronic device when the receiving module 31 receives a second input to the first control. The receiving module 31 is specifically used to receive a first voice message sent by the second electronic device.

[0198] In an embodiment of the present application, a communication device is provided. After receiving the voice of the user of the second electronic device, the communication device can display the voice to the user of the communication device in the form of text, so that the user of the communication device can understand the content of the call of the user of the second electronic device, so that the user of the communication device can directly make a text reply based on the displayed text information, and then the communication device can convey the text information replied by the user of the communication device to the user of the second electronic device in the form of voice with a set first tone, so that the user of the second electronic device can understand the content of the call of the user of the communication device. In this way, through this voice-text-voice communication mode, even if the environment where the user is located is not convenient for voice calls, it can respond to the content of the call of the user of the second electronic device in real time, thereby improving the real-time and smoothness of the voice call of the communication device, thereby improving the interactive experience of the user during the call.

[0199] The communication device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0200] The communication device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0201] The communication device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0202] Alternatively, if Fig.14As shown, an embodiment of the present application further provides an electronic device 1000, including a processor 1001 and a memory 1002, wherein the memory 1002 stores programs or instructions that can be executed on the processor 1001, and when the program or instructions are executed by the processor 1001, the various steps of the above-mentioned call method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0203] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0204] Fig.15 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0205] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0206] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.15 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0207] The radio frequency unit 101 is used to receive the first voice information sent by the second electronic device when the first electronic device is talking with the second electronic device.

[0208] The processor 110 is configured to convert the first voice information into first text information.

[0209] The display unit 106 is used to display the first text information.

[0210] The user input unit 107 is used to obtain second text information input by the user.

[0211] The processor 110 is further configured to convert the second text information into second voice information based on the first timbre.

[0212] The radio frequency unit 101 is further configured to send the second voice information to the second electronic device.

[0213] Optionally, the display unit 106 is specifically used to display a first interface, which includes a text display area and a text input area; the first text information is displayed in the text display area. The user input unit 107 is used to receive a first input of the second text information by the user in the text input area. The display unit 106 is also used to display the second text information in the text display area in response to the first input. The input unit 104 is also used to obtain the second text information from the text display area.

[0214] Optionally, the first interface further includes a text pending area. The display unit 106 is further configured to display the second text information in the text pending area before the second text information is displayed in the text display area. The display unit 106 is specifically configured to update the second text information from the text pending area to the text display area for display when the correlation between the second text information in the text pending area and the first text information is greater than or equal to a first threshold.

[0215] Optionally, the above-mentioned processor 110 is also used to, after displaying the second text information in the text pending area, when the correlation between the second text information in the text pending area and the first text information is less than a first threshold, if it is detected that text information is added to the text pending area within a first time length, then determine whether the correlation between the updated text information in the text pending area and the first text information is greater than or equal to the first threshold.

[0216] Optionally, the display unit 106 is specifically configured to share the first interface to the third electronic device for display when the first electronic device is interconnected with the third electronic device.

[0217] Optionally, the user input unit 107 is further used to receive a third input from the user to the timbre selection control of the first interface. The display unit 106 is further used to display the timbre selection interface in response to the third input. The user input unit 107 is further used to receive a first timbre selected by the user in the timbre selection interface. The processor 110 is specifically used to convert the second text information based on the first timbre to obtain the second voice information. The timbre selection interface includes at least one of the following: an online timbre library, a preset timbre library, and a timbre library synthesized based on the voiceprint of at least one user and languages ​​of different language types.

[0218] Optionally, the display unit 106 is further used to display a first control and a second control before the first electronic device establishes a call with the second electronic device, upon receiving a call request from the second electronic device, wherein the first control is used to indicate the selection of a call mode for converting between voice and text, and the second control is used to indicate the selection of a voice call mode. The processor 110 is further used to establish a call connection between the first electronic device and the second electronic device upon receiving a second input to the first control. The radio frequency unit 101 is specifically used to receive a first voice message sent by the second electronic device.

[0219] In an embodiment of the present application, an electronic device is provided. After receiving the voice of the user of the second electronic device, the electronic device can display the voice to the user of the electronic device in the form of text, so that the user of the electronic device can understand the content of the call of the user of the second electronic device, so that the user of the electronic device can directly make a text reply based on the displayed text information, and then the electronic device can convey the content of the reply to the user of the second electronic device in the form of voice with a set first tone, so that the user of the second electronic device can understand the content of the call of the user of the electronic device. In this way, through this voice-text-voice call mode, even if the environment where the user is located is not convenient for voice calls, it can respond to the call content of the user of the second electronic device in real time, thereby improving the real-time and smoothness of the electronic device in voice calls, thereby improving the user's interactive experience during the call.

[0220] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and can achieve the same technical effect. To avoid repetition, it is not repeated here. The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, it is not repeated here.

[0221] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0222] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0223] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0224] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned call method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0225] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0226] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned call method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0227] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0228] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned call method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0229] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0230] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0231] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for making a call, characterized in that: Applicable to a first electronic device, comprising: When the first electronic device is talking to the second electronic device, receiving a first voice message sent by the second electronic device; Convert the first voice information into first text information, and display the first text information; Obtaining second text information input by the user; The second text information is converted into second voice information based on the first timbre, and the second voice information is sent to the second electronic device.

2. The method according to claim 1, characterized in that The displaying of the first text information comprises: Display a first interface, wherein the first interface includes a text display area and a text input area; Displaying the first text information in the text display area; The step of obtaining the second text information input by the user includes: receiving a first input of the user to the second text information in the text input area; In response to the first input, displaying the second text information in the text display area; The second text information is acquired from the text display area.

3. The method according to claim 2, characterized in that The first interface also includes a text pending area; Before displaying the second text information in the text display area, the method further includes: Displaying the second text information in the text pending area; Displaying the second text information in the text display area includes: When the correlation between the second text information in the text pending determination area and the first text information is greater than or equal to a first threshold, the second text information is updated from the text pending determination area to the text display area for display.

4. The method according to claim 3, characterized in that After the second text information is displayed in the text pending area, the method further includes: In the case where the correlation between the second text information in the text pending area and the first text information is less than the first threshold, if it is detected that text information is added to the text pending area within a first time period, it is determined whether the correlation between the updated text information in the text pending area and the first text information is greater than or equal to the first threshold.

5. The method according to claim 2, characterized in that: The displaying of the first interface includes: When the first electronic device is interconnected with a third electronic device, the first interface is shared with the third electronic device for display.

6. The method according to claim 2, characterized in that The converting the second text information into second voice information based on the first timbre includes: Receiving a third input from a user to the timbre selection control of the first interface; In response to the third input, displaying a tone selection interface; Receive a first timbre selected by a user in the timbre selection interface; Convert the second text information based on the first timbre to obtain the second voice information; The timbre selection interface includes at least one of the following: an online timbre library, a preset timbre library, and a timbre library synthesized based on the voiceprint of at least one user and languages ​​of different language types.

7. The method according to any one of claims 1 to 5, characterized in that Before the first electronic device establishes a call with the second electronic device, the method further includes: In the case of receiving a call request from the second electronic device, displaying a first control and a second control, wherein the first control is used to indicate selection of a call mode of converting between voice and text, and the second control is used to indicate selection of a voice call mode; The step of receiving the first voice information sent by the second electronic device when the first electronic device is talking with the second electronic device includes: When a second input to the first control is received, a call connection between the first electronic device and the second electronic device is established, and a first voice message sent by the second electronic device is received.

8. A communication device, characterized in that: include: Receiving module, processing module, display module and sending module; The receiving module is used to receive the first voice information sent by the second electronic device when the first electronic device is talking with the second electronic device; The processing module is used to convert the first voice information received by the receiving module into first text information; The display module is used to display the first text information obtained by the processing module; The receiving module is further used to obtain second text information input by the user; The processing module is further configured to convert the second text information acquired by the receiving module into second voice information based on the first timbre; The sending module is used to send the second voice information obtained by the processing module to the second electronic device.

9. The device according to claim 8, characterized in that The display module is specifically used to display a first interface, the first interface includes a text display area and a text input area; the first text information is displayed in the text display area; The receiving module is used to receive a first input of the user to the second text information in the text input area; The display module is further configured to display the second text information in the text display area in response to the first input received by the receiving module; The receiving module is specifically configured to obtain the second text information from the text display area.

10. The device according to claim 9, characterized in that The first interface also includes a text pending area; The display module is further configured to display the second text information in the text pending area before displaying the second text information in the text display area; The display module is specifically configured to update the second text information from the text pending area to the text display area for display when the correlation between the second text information and the first text information in the text pending area is greater than or equal to a first threshold.

11. The device according to claim 10, characterized in that The processing module is also used to determine whether the correlation between the updated text information in the text pending area and the first text information is greater than or equal to the first threshold if, after the display module displays the second text information in the text pending area, the correlation between the second text information in the text pending area and the first text information is less than the first threshold, if it is detected that text information is added to the text pending area within a first time period.

12. The device according to claim 9, characterized in that The display module is specifically used to share the first interface to the third electronic device for display when the first electronic device is interconnected with the third electronic device.

13. The device according to claim 9, characterized in that The receiving module is further used to receive a third input from the user to the timbre selection control of the first interface; The display module is further configured to display a tone selection interface in response to the third input received by the receiving module; The receiving module is further used to receive a first timbre selected by a user in the timbre selection interface; The processing module is specifically used to convert the second text information based on the first timbre to obtain second voice information; The timbre selection interface includes at least one of the following: an online timbre library, a preset timbre library, and a timbre library synthesized based on the voiceprint of at least one user and languages ​​of different language types.

14. The device according to any one of claims 8 to 12, characterized in that The display module is further used to display a first control and a second control when a call request from the second electronic device is received before the first electronic device establishes a call with the second electronic device, wherein the first control is used to indicate the selection of a call mode for converting between voice and text, and the second control is used to indicate the selection of a voice call mode; The processing module is further configured to establish a call connection between the first electronic device and the second electronic device when the receiving module receives a second input to the first control; The receiving module is specifically configured to receive the first voice information sent by the second electronic device.

15. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the call method according to any one of claims 1 to 7 are implemented.

16. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the communication method according to any one of claims 1 to 7 are implemented.

Images