Voice processing method, smart watch, storage medium and program product

Abstract

The embodiment of the invention discloses a voice processing method, a smart watch, a storage medium and a program product, which are used for displaying current key content information obtained by carrying out key information extraction on current sound information on the smart watch when communicating with an opposite user so as to assist a current user in understanding communication content of the opposite user. The method is applied to the smart watch, the smart watch is provided with a display device, and the method comprises the following steps: acquiring current sound information; current communication content information corresponding to the current sound information is subjected to key content information extraction, current key content information is obtained, and the current key content information is simplified information used for expressing the current communication content information corresponding to the current sound information; and displaying the current key content information on a first display interface of the display device.

Description

Technical Field

[0001] This application relates to the field of wearable devices, and more particularly to a voice processing method, a smartwatch, a storage medium, and a program product. Background Technology

[0002] Traditional hearing aids typically only provide sound gain compensation to help hearing-impaired users hear surrounding sounds more clearly. However, when faced with someone speaking quickly and at length, even with gain compensation, users may still struggle to understand everything, especially in complex communication scenarios. In such situations, users may need to repeatedly ask the other person questions or request them to slow down, severely impacting communication efficiency and the overall experience. Summary of the Invention

[0003] This application provides a voice processing method, a smartwatch, a storage medium, and a program product, which can display key content information obtained by extracting key information from the current voice information on the smartwatch when communicating with another user, thereby assisting the current user in understanding the other user's communication content.

[0004] A second aspect of this application provides a voice processing method applied to a smartwatch, the smartwatch being equipped with a display device, the method comprising:

[0005] Get the current sound information;

[0006] The current communication content information corresponding to the current sound information is processed by extracting key content information to obtain current key content information, which is simplified information used to express the current communication content information corresponding to the current sound information;

[0007] The current key content information is displayed on the first display interface of the display device.

[0008] A second aspect of this application provides a smartwatch, the smartwatch being equipped with a display device, and the smartwatch further comprising:

[0009] A processing device is used to acquire current sound information; extract key content information from the current communication content information corresponding to the current sound information to obtain current key content information, wherein the current key content information is simplified information used to express the current communication content information corresponding to the current sound information; and display the current key content information on the first display interface of the display device.

[0010] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect of this application.

[0011] A fourth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect of this application.

[0012] The fifth aspect of this application discloses an application publishing platform for publishing computer program products, wherein when the computer program products are run on a computer, the computer executes the method described in the first aspect of this application.

[0013] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0014] In this embodiment, current voice information is acquired; key content information is extracted from the current communication content information corresponding to the current voice information to obtain current key content information, which is simplified information used to express the current communication content information corresponding to the current voice information; the current key content information is displayed on the first display interface of the display device. This allows the user to display the current key content information obtained by extracting key information from the current voice information on a smartwatch when communicating with another user, thereby assisting the current user in understanding the other user's communication content. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments and the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and other drawings can be obtained based on these drawings.

[0016] Figure 1A This is a schematic diagram of one embodiment of the smartwatch in this application.

[0017] Figure 1B This is a schematic diagram of one embodiment of the headphones and smartwatch in this application;

[0018] Figure 1C This is a schematic diagram of an embodiment of removing the earphones from the smartwatch in this application.

[0019] Figure 1D This is a schematic diagram of a scenario in which an embodiment of this application is applied;

[0020] Figure 1E This is a schematic diagram of a scenario in which an embodiment of this application is applied;

[0021] Figure 1F This is a schematic diagram illustrating another scenario in which the embodiments of this application are applied;

[0022] Figure 2This is a schematic diagram of one embodiment of the speech processing method in this application;

[0023] Figure 3A This is a schematic diagram illustrating the display interface of a smartwatch in this embodiment of the present application, which includes at least two simplified options;

[0024] Figure 3B This is a schematic diagram of the reference key content information corresponding to the first simplified option in the embodiments of this application;

[0025] Figure 3C This is a schematic diagram of the reference key content information corresponding to the second simplified option in the embodiments of this application;

[0026] Figure 3D This is a schematic diagram illustrating the display of current key content information on the display interface of a smartwatch in an embodiment of this application;

[0027] Figure 4A This is a schematic diagram of the headphones including a gain function button in an embodiment of this application;

[0028] Figure 4B This is a schematic diagram of the display interface of a smartwatch in an embodiment of this application, including a gain function control;

[0029] Figure 4C This is a schematic diagram illustrating one embodiment of a user viewing communication content in this application.

[0030] Figure 5 This is a schematic diagram of another embodiment of the speech processing method in this application;

[0031] Figure 6 This is a schematic diagram of one embodiment of the smartwatch in this application.

[0032] Figure 7 This is a schematic diagram of another embodiment of the smartwatch in this application. Detailed Implementation

[0033] This application provides a voice processing method, a smartwatch, a storage medium, and a program product, which can display key content information obtained by extracting key information from the current voice information on the smartwatch when communicating with another user, thereby assisting the current user in understanding the other user's communication content.

[0034] To enable those skilled in the art to better understand the present application, the technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. All embodiments based on the present application should fall within the scope of protection of the present application.

[0035] 1. Speech recognition algorithms, also known as Automatic Speech Recognition (ASR), aim to convert human speech into text. This technology has wide applications in various fields, including real-time online speech recognition and offline speech recognition. Real-time online speech recognition involves converting a continuous stream of speech into text in real time, while offline speech recognition involves recognizing an existing audio file as a whole.

[0036] The basic process of speech recognition includes preprocessing, feature extraction, the establishment of acoustic and language models, and the final decoding process. In the preprocessing stage, the raw audio signal is sampled, quantized, and encoded to convert it into a digital signal. Subsequently, speech features, such as Mel Frequency Cepstrum Coefficients (MFCCs) and Filter Bank (FBank) features, are extracted using methods such as the Fast Fourier Transform (FFT).

[0037] Traditional speech recognition models rely on a combination of acoustic and language models, using statistical models such as Hidden Markov Models (HMMs) to identify phonemes and words. Modern end-to-end speech recognition models, such as Transformer-based models, Conformer, and Paraformer, directly predict text from the input audio or its features, eliminating some of the intermediate steps found in traditional models.

[0038] In devices such as smartwatches, voice recognition technology enables a variety of convenient functions, such as making phone calls, sending messages, and checking the weather via voice commands. The technical details behind these functions include voice acquisition, preprocessing, deep learning algorithms for recognition, and executing corresponding operations based on the recognition results.

[0039] The development of speech recognition technology has evolved from simple finite state automata to deep learning methods. With technological advancements, the accuracy of speech recognition and the diversity of its application scenarios have continuously improved. In the future, speech recognition technology is expected to achieve further breakthroughs in multimodal fusion, cross-language recognition, low-power recognition, and personalized recognition, and will be more widely applied in smart homes, healthcare, education, and other fields.

[0040] 2. Natural Language Processing (NLP) is an important area of ​​artificial intelligence, focusing on how computers and humans interact through human language. NLP tasks are mainly divided into four categories: category-to-sequence, sequence-to-category, synchronous sequence-to-sequence, and asynchronous sequence-to-sequence. These tasks are essentially processes of converting one data type into another, similar to or the same as machine learning models. NLP preprocessing steps include corpus collection, text cleaning, word segmentation, stop word removal, standardization, and feature extraction. In English NLP, preprocessing steps include word segmentation, stemming, lemmatization, part-of-speech tagging, named entity recognition, and chunking; while in Chinese NLP, the steps include Chinese word segmentation, part-of-speech tagging, named entity recognition, and stop word removal.

[0041] The core tasks of NLP include Natural Language Understanding (NLU) and Natural Language Generation (NLG). NLU focuses on enabling computers to understand the nuances of human language, including context, intention, emotion, and ambiguity. NLG, on the other hand, focuses on creating language similar to human language based on databases or rule sets, with the goal of generating text that is easily understood by humans.

[0042] The challenges facing NLP include language diversity, ambiguity, robustness, knowledge dependence, and context dependence. Despite these challenges, NLP has wide applications in many fields, such as sentiment analysis, chatbots, speech recognition, and machine translation.

[0043] In terms of technical implementation, NLP can use either traditional machine learning methods or deep learning methods. Traditional methods focus on rule-based processing and statistical approaches, while deep learning methods focus on using neural networks and transformer models, such as BERT and ChatGPT, which utilize attention mechanisms to improve the ability to process natural language.

[0044] The development of NLP initially relied on rule-based processing, later evolving to utilize statistical methods and machine learning algorithms. With the advancement of deep learning technologies, particularly the emergence of neural networks and transformer models, the field of NLP has made significant progress. The future of NLP is expected to see integration with speech recognition, computer vision, and robotics, giving rise to more advanced and complex systems. NLP systems will also become more personalized, better understanding individual users and adjusting their responses and suggestions accordingly.

[0045] This application provides a speech processing method, which can be a speech information summarization and hearing aid method. Specifically, when the other user speaks quickly and the content is lengthy, the smartwatch can capture the other user's speech information in real time, analyze it with AI, summarize it briefly, and display it on the smartwatch screen to help the user better understand the other user's speech. Optionally, it can also output speech information through headphones, providing users with more intelligent and convenient speech understanding assistance in noisy or complex communication scenarios. Figure 1A The image shown is a schematic diagram of one embodiment of the smartwatch in this application.

[0046] like Figure 1B The diagram shown is a schematic representation of one embodiment of the headphones and smartwatch in this application. The headphones and smartwatch in this application embodiment can be a single, integrated pair or separate devices. If the headphones and smartwatch are a single, integrated pair, the headphones can be plugged into the smartwatch for charging when the battery is low or dead. The headphones can be over-ear headphones, in-ear headphones, neckband headphones, true wireless headphones, ear-hook headphones, etc., and this application embodiment does not impose specific limitations.

[0047] like Figure 1C The diagram shown illustrates an embodiment of this application where the earphones are removed from the smartwatch. If the earphones and smartwatch are a complete set, the user can remove the earphones for wearing and use; then, the earphones can be used to output the voice information of the other user, and the smartwatch can display the key content information corresponding to the other user's voice information.

[0048] like Figure 1D The diagram shown is a schematic representation of a scenario in which this application embodiment is applied. Figure 1D In the scenario shown, user A is wearing a smartwatch, and user B and user A are communicating. The smartwatch is equipped with a display device that can display the current communication content information corresponding to the current voice information spoken by user B. After extracting key content information, the current key content information is obtained to help user A understand what user B is saying.

[0049] like Figure 1E The diagram shown is a schematic representation of a scenario in which this application embodiment is applied. Figure 1EIn the scenario depicted, User A is wearing headphones and a smartwatch. User B and User A are conversing. The smartwatch has a display device that shows the current communication content, extracted from the key information extracted from User B's current voice message. Simultaneously, User A can also hear User B's voice through the headphones to help User A understand what User B is saying. It should be noted that the number of users conversing with User A can be one or more; this embodiment does not impose a specific limitation.

[0050] like Figure 1F The diagram shown illustrates another scenario in which an embodiment of this application is used. Figure 1F In the scenario depicted, User A is wearing headphones and a smartwatch and is located at location A. User A is having a voice or video call with User B, who is also located at location B, via the smartwatch and headphones. The smartwatch has a display device that shows the current communication content, extracted from the key information of User B's speech. Simultaneously, User A can also hear User B's voice through the headphones to help User A understand what User B is saying.

[0051] The following is in conjunction with the above. Figures 1A-1F The smartwatch, earphones, and application scenarios shown in the illustration further illustrate the embodiments of this application, such as... Figure 2 The diagram shown is a schematic representation of an embodiment of the voice processing method in this application, applied to a smartwatch. The smartwatch is equipped with a display device, and the method may include:

[0052] 201. Obtain current sound information.

[0053] Optionally, in some embodiments of this application, obtaining the current sound information may include: receiving the current sound information sent by the earphone, wherein the smartwatch is connected to the earphone; or, detecting the current sound information.

[0054] For example, during the current user's communication with the other user, the communication method can be direct dialogue, or it can be voice call, video call, conference call, etc. This application embodiment does not make specific limitations.

[0055] In conversational communication, if the user is not wearing headphones but is wearing a smartwatch, the smartwatch can detect the current sound information. If the user is wearing both headphones and a smartwatch, and the headphones and smartwatch are connected, the smartwatch can detect the current sound information, and the headphones can detect the current sound information and send it to the smartwatch, which will then receive the sound information sent by the headphones.

[0056] During voice, video, or conference calls, if the user is not wearing headphones but is wearing a smartwatch, the smartwatch can detect the current audio information. If the user is wearing both headphones and a smartwatch, and the voice, video, or conference call is conducted through the smartwatch, the smartwatch can detect the current audio information. If the user is wearing both headphones and a smartwatch, and the voice, video, or conference call is conducted through another electronic device that is connected to both the headphones and the smartwatch, the headphones can detect the current audio information and send it to the smartwatch, which then receives the audio information sent by the headphones.

[0057] In this technical solution, the smartwatch can detect the current sound information itself or receive the current sound information sent by the headphones. This application does not specifically limit the specific implementation, thus improving the feasibility of the solution. In the implementation where the smartwatch receives the current sound information sent by the headphones, the headphones and smartwatch are in a connected state.

[0058] Optionally, in some embodiments of this application, the detection of the current sound information may include: detecting the current sound information through a microphone.

[0059] In this technical solution, the smartwatch primarily relies on its built-in microphone to detect ambient sounds. When the user is wearing the smartwatch and communicating with another user, the smartwatch's microphone detects external sounds, thus improving the feasibility of the solution.

[0060] Optionally, in some embodiments of this application, the current sound information is not the sound information of the current user, and the current user is the user currently using the smartwatch.

[0061] In this technical solution, acquiring the current voice information is for the purpose of subsequently extracting key content information from the current communication content corresponding to the current voice information, obtaining current key content information, and then displaying this key information on the first display interface through a display device to help the current user understand the other user's communication content. If the current voice information is the current user's own voice information, then displaying the current key content information corresponding to the current user's own current voice information is not very meaningful, as the current user generally knows what they have said. The main purpose is to obtain the current key content information corresponding to the current voice information of the other user with whom one is communicating, thereby improving the efficiency and quality of communication.

[0062] Optionally, in some embodiments of this application, the current voice information is obtained through voiceprint recognition technology, which is not the voice information of the current user.

[0063] In this technical solution, voiceprint recognition technology can be used to distinguish between the voice information of the current user and that of non-current users. For example, the voiceprint of the current user can be pre-set in the headphones. The headphones perform voiceprint matching on the current voice information. If it is the current user's voice, it is saved; otherwise, subsequent processing such as extracting key content information is performed. Thus, the smartwatch only extracts key content information from the voice information of non-current users, obtains and displays this key content information, making it easier for the current user to obtain the key content information corresponding to the other user's current voice information, thereby improving the efficiency and quality of communication.

[0064] 202. The current communication content information corresponding to the current sound information is extracted by extracting key content information to obtain current key content information, which is simplified information used to express the current communication content information corresponding to the current sound information.

[0065] Optionally, in some embodiments of this application, the step of extracting key content information from the current communication content information corresponding to the current sound information to obtain current key content information may include: extracting key content information from the current communication content information corresponding to the current sound information through an artificial intelligence processing unit to obtain current key content information.

[0066] In this technical solution, the AI ​​processing unit in the smartwatch can extract key content information from the acquired current voice information, and then display this key content information on the first display interface of the display device. This helps the user understand the other party's voice information in the current call. This not only enhances comprehension—by extracting key content information from the current voice information using AI, the user can quickly understand the other party's intent even if they speak quickly or the content is complex—but also improves communication efficiency: the user doesn't need to repeatedly ask or request the other party to slow down; they can simply follow the conversation by viewing the key content information on the smartwatch screen.

[0067] Optionally, in some embodiments of this application, the step of extracting key content information from the current communication content information corresponding to the current sound information through the artificial intelligence processing unit to obtain current key content information may include: extracting key content information from the current communication content information corresponding to the current sound information through a preset speech recognition algorithm and natural language processing algorithm in the artificial intelligence processing unit to obtain current key content information.

[0068] Example 1: Suppose the current user is User A, and the other user communicating with User A is User B. User A has hearing loss and wears a smartwatch. User A and User B are having a conversation. User B says to User A: "Yesterday I went to the supermarket and bought a lot of things, including some fresh fruits and vegetables, as well as some snacks and drinks. When I got back, I washed all the fruits, put the vegetables in the refrigerator, and put the snacks and drinks in the kitchen cabinet. Oh, and I also bought a few bottles of milk; remember to drink them before they expire." Because User A has hearing loss, and User B's current speech content, i.e., the current voice information, is relatively large, User A's smartwatch can detect User B's current voice information and use a preset speech recognition algorithm and natural language processing algorithm to extract key content information, such as: "Bought fruits, vegetables, snacks, and milk; remember to drink the milk after putting them away." This is then displayed on the smartwatch's screen. User A only needs to raise their wrist to see the key content information corresponding to User B's current voice information. This method is particularly useful when the current user is communicating with another user, and the current user has hearing impairment, and / or the other user speaks quickly or has lengthy content, as it can help the current user keep up with the other user's understanding.

[0069] Example 2: For example, the current user is user A, and the other user communicating with user A is user B. User A has hearing loss and is wearing headphones and a smartwatch. User A and user B are having a conversation. User B says to User A, "Yesterday I went to the supermarket and bought a lot of things, including some fresh fruits and vegetables, as well as some snacks and drinks. When I got back, I washed all the fruits, put the vegetables in the refrigerator, and put the snacks and drinks in the kitchen cabinet. Oh, and I also bought a few bottles of milk; remember to drink them before they expire." Because User A has hearing loss, and User B's current speech content, i.e., the current voice information, is quite extensive, although User A's headphones can acquire and output User B's current voice information, they may not be able to effectively acquire the key content information. However, User A's smartwatch can detect User B's current voice information and use preset speech recognition and natural language processing algorithms to extract the key content information, such as: "I bought fruits, vegetables, snacks, and milk; remember to drink the milk after you put them away." This information is then displayed on the smartwatch's screen. User A only needs to raise their wrist to see the key content information corresponding to User B's current voice information. This method is particularly useful when the current user is communicating with another user, and the current user has hearing impairment, and / or the other user speaks quickly or has lengthy content, as it can help the current user keep up with the other user's understanding.

[0070] In this technical solution, the smartwatch can use the preset speech recognition algorithm and natural language processing algorithm in the artificial intelligence processing unit to extract key content information from the current sound information, thereby improving the accuracy of the current key content information extraction.

[0071] Optionally, in some embodiments of this application, the method may further include: upon detecting a simplified setting trigger command, displaying a fourth display interface through the display device, the fourth display interface including at least one simplified option; when at least two simplified options are included, different simplified options extract key content information from the communication content information, resulting in different degrees of simplification of the key content information.

[0072] The step of extracting key content information from the current communication content information corresponding to the current voice information to obtain current key content information may include: when a selection instruction for a target simplification option is detected, using the simplification level corresponding to the target simplification option, extracting key content information from the current communication content information corresponding to the current voice information to obtain current key content information, wherein the at least two simplification options include the target simplification option.

[0073] Optionally, in some embodiments of this application, the at least two simplification options include a first simplification option and a second simplification option, wherein the degree of simplification of the second simplification option is greater than that of the first simplification option; the target simplification option is either the first simplification option or the second simplification option.

[0074] For example, the simplified settings trigger instruction mentioned above can be triggered by the current user performing a touch operation on the simplified settings control (e.g., pressing, double-clicking, swiping, etc.), or by the current user triggering it via voice; the selection instruction for the target simplified option mentioned above can be triggered by the current user performing a touch operation on the target simplified option control (e.g., pressing, double-clicking, swiping, etc.), or by the current user triggering it via voice. The specific embodiments of this application are not limited.

[0075] In this technical solution, users can use a smartwatch to select the level of simplification for the audio information. Different simplification options offer varying degrees of simplification, allowing users to choose the option that best suits their needs and helps them better understand the communication content. Users can adjust the AI ​​simplification level through the smartwatch's touchscreen interface or voice commands.

[0076] Optionally, in some embodiments of this application, the third display interface further includes reference communication content information and at least one reference key content information, the at least one reference key content information corresponding to the at least one simplification option; the target reference key content information is used to indicate the reference communication content information after simplification using the degree of simplification corresponding to the target simplification option; wherein, the at least one reference key content information includes the target reference key content information.

[0077] It should be noted that the reference communication content information and at least one reference key content information can be displayed on the same display interface or on different display interfaces, depending on the actual needs. This application embodiment does not make specific limitations.

[0078] For example, such as Figure 3A The diagram shown illustrates a smartwatch display interface with at least two simplified options in an embodiment of this application. For example, the reference communication content information is: "I like to eat lots and lots of delicious food, such as: chive and egg dumplings, egg fried noodles, fried rice, freshly baked bread, and chilled milk."

[0079] If the current user selects the first simplified option, then the corresponding key reference information is: I like to eat chive and egg dumplings, egg fried noodles, fried rice, freshly baked bread, and chilled milk. For example... Figure 3BThe diagram shown is a schematic representation of the reference key content information corresponding to the first simplified option in the embodiments of this application.

[0080] If the current user selects the second simplified option, then the corresponding key reference information is: I like to eat dumplings, noodles, rice, bread, and milk. For example... Figure 3C The diagram shown is a schematic representation of the reference key content information corresponding to the second simplified option in the embodiments of this application.

[0081] In this technical solution, at least two simplification options can be included. These options extract key content information from the audio data and describe the degree of simplification. Different simplification options selected by the user correspond to different levels of simplification. This allows the user to choose a simplification option that suits their needs, subsequently helping them better understand the communication content.

[0082] 203. Display the current key content information on the first display interface of the display device.

[0083] For example, such as Figure 3D The image shown is a schematic diagram illustrating the display of current key content information on the smartwatch's screen in an embodiment of this application. For example, the current key content information is: "I bought fruits, vegetables, snacks, and milk. Remember to drink the milk after you put them away."

[0084] Optionally, in some embodiments of this application, when a language setting trigger command is detected, a fifth display interface is displayed through the display device. The fifth display interface includes at least one language option. When at least two language options are included, different language options correspond to different language types for displaying the current key content information.

[0085] Displaying the current key content information on the first display interface of the display device may include: upon detecting a selection instruction for a target language option, displaying the current key content information on the first display interface of the display device using the language type corresponding to the target language option.

[0086] For example, the language setting trigger instruction mentioned above can be triggered by the current user performing a touch operation on the language setting control (e.g., pressing, double-clicking, swiping, etc.), or by the current user triggering it via voice; the above-mentioned instruction to select the target language option can be triggered by the current user performing a touch operation on the target language option control (e.g., pressing, double-clicking, swiping, etc.), or by the current user triggering it via voice. The specific embodiments of this application are not limited.

[0087] In this technical solution, users can use a smartwatch to select the language type for key content information. Different language options offer different language types, allowing users to choose the language type that best suits their needs and helps them better understand the communication content. Users can adjust the AI-simplified language type through the smartwatch's touchscreen interface or voice commands.

[0088] Optionally, in some embodiments of this application, when a display format trigger command is detected, a sixth display interface is displayed through the display device. The sixth display interface includes at least one display format option. When at least two display format options are included, the display formats of the current key content information are different for different display format options.

[0089] Displaying the current key content information on the first display interface of the display device may include: upon detecting a selection instruction for a target display format option, displaying the current key content information on the first display interface of the display device using the language type corresponding to the target display format option.

[0090] For example, at least two display options include font size, whether to bold, whether to italic, whether to underline, font color, different colors to highlight text, whether to use graphic elements, etc., and the specific embodiments of this application are not limited.

[0091] The aforementioned display format triggering instruction can be triggered by the current user performing a touch operation (e.g., pressing, double-clicking, swiping, etc.) on the display format setting control, or by the current user triggering it via voice. The aforementioned instruction for selecting the target display format option can be triggered by the current user performing a touch operation (e.g., pressing, double-clicking, swiping, etc.) on the target display format option control, or by the current user triggering it via voice. The specific implementation of this application is not limited.

[0092] In this technical solution, users can use a smartwatch to select the display format for key information. Different display options offer different presentation styles, allowing users to choose the format that best suits their needs and helps them better understand the communication content. Users can adjust the display format of key information through the smartwatch's touchscreen interface or voice commands.

[0093] It should be noted that the above-mentioned at least one simplification option, at least one language option, and at least one display format option can be displayed on the same interface or on different interfaces. The specific layout can be determined according to the size of the display interface or the user's usage habits, etc. This application embodiment does not make specific limitations.

[0094] Optionally, in some embodiments of this application, before the current key content information is obtained by extracting key content information from the current communication content information corresponding to the current voice information and displaying the current key content information on the first display interface of the display device, the method may further include: responding to the opening operation of the function control for extracting and displaying key content information in the voice information, and enabling the function for extracting and displaying key content information.

[0095] Optionally, in some embodiments of this application, after extracting key content information from the current communication content information corresponding to the current voice information to obtain current key content information, and displaying the current key content information on the first display interface of the display device, the method may further include: responding to a closing operation of the function control for extracting and displaying key content information in the voice information, and disabling the function for extracting and displaying key content information.

[0096] For example, when a user needs to communicate with another user, due to a noisy environment or hearing impairment, a smartwatch may be needed to display key information during the communication to help the user understand the content. In this case, the function of extracting and displaying key information can be enabled on the smartwatch. If the user does not need the function of extracting and displaying key information, or if other functions need to be performed on the smartwatch, the function can be disabled to avoid interference with other functions.

[0097] In this technical solution, users can enable or disable the function of extracting and displaying key content information on their smartwatches according to their actual needs, thereby improving the user experience.

[0098] In one use case, the current user is wearing both a smartwatch and headphones, and the headphones acquire and output the current sound information.

[0099] For example, the headset may include one or a pair of earphones. When a user is wearing both a smartwatch and headset simultaneously, the smartwatch and headset may or may not be connected. If information needs to be exchanged between the smartwatch and headset, they need to be connected; for example, the smartwatch and headset can connect via Bluetooth, enabling them to receive and process each other's voice information in real time. If information exchange is not required between the smartwatch and headset, they may or may not be connected.

[0100] In this technical solution, the current audio information refers to audio information that is not that of the current user. If the current audio information were that of the current user, then outputting the current user's own audio information through their own headphones would be largely meaningless for the communication process. Therefore, outputting the audio information of the other user being communicated with through the current user's own headphones helps the current user obtain the other user's current audio information, thereby improving the efficiency and quality of communication.

[0101] Optionally, in some embodiments of this application, the acquisition of the current sound information by the headphones may include: the headphones detecting and obtaining the current sound information; or,

[0102] The earphone receives the current sound information sent by the smartwatch; that is, the smartwatch sends the current sound information to the earphone, and the current sound information is used for output by the earphone, wherein the smartwatch and the earphone are connected.

[0103] In this technical solution, the headphones can detect the current sound information themselves, or the smartwatch and headphones can be connected, with the headphones receiving the current sound information sent by the smartwatch. Specific embodiments of this application are not limited to this. The headphones can also simultaneously output the current sound information, allowing the user to obtain the current sound information through the headphones and also view the corresponding key information on the smartwatch's display interface, thereby improving the efficiency and quality of communication.

[0104] Optionally, in some embodiments of this application, the detection of the current sound information by the headphones may include: detecting the current sound information through the microphone of the headphones.

[0105] In this technical solution, the headphones primarily rely on a built-in microphone to capture ambient sounds when detecting sound information. When a user is wearing the headphones and communicating, the microphone detects the current external sound information and outputs it, thus improving the feasibility of the solution.

[0106] Optionally, in some embodiments of this application, the current voice information is obtained through voiceprint recognition technology, which is not the voice information of the current user.

[0107] In this technical solution, voiceprint recognition technology can be used to distinguish between the voice information of the current user and that of non-current users. For example, the voiceprint of the current user can be pre-set in the headphones. The headphones perform voiceprint matching on the current voice information. If the voiceprint matches the current user's voice, it is not played; otherwise, it is played. This ensures that only the voice information of non-current users is played on the headphones, making it easier for the current user to obtain the other party's voice information, thereby improving the efficiency and quality of communication.

[0108] Optionally, in some embodiments of this application, the method may further include: the smartwatch performing gain processing on the current sound information to obtain gain-processed current sound information; the smartwatch sending the gain-processed current sound information to the headphones, so that the headphones can output the gain-processed current sound information.

[0109] In this technical solution, for users with hearing impairments, or in noisy environments, the smartwatch can perform gain processing on the current sound information to obtain the amplified sound information, which is then sent to the headphones. The headphones output the amplified sound information, helping the user hear the other user's voice more clearly. Through these gain processing methods, the current sound information can be effectively amplified according to the user's specific needs and application scenarios to improve sound quality and listening experience.

[0110] Optionally, in some embodiments of this application, the acquisition and output of current sound information by the headphones may include: acquiring the current sound information through the headphones; performing gain processing on the current sound information to obtain gain-processed current sound information; and outputting the gain-processed current sound information through the headphones.

[0111] For example, the implementation methods for gain processing of the current sound information typically include, but are not limited to, the following:

[0112] (1) Volume gain:

[0113] Volume gain algorithms can be used to increase or decrease the amplitude of an audio signal as needed. For example, a power function can be used to apply gain to the audio signal to achieve non-uniform volume amplification. This method is commonly implemented in audio editing software, such as the `volume_augment` function described in the `speech_data_augment` project on GitHub.

[0114] (2) Speed ​​increase:

[0115] Speed ​​gain involves changing the speed of audio playback while keeping the pitch constant. This can be achieved by interpolating the audio signal, such as using linear interpolation with the NumPy library, thus adjusting the speed.

[0116] (3) Noise enhancement:

[0117] In some cases, it may be necessary to add noise to simulate sound effects in a real environment. This can be achieved by adding natural or artificial noise to the original audio signal.

[0118] (4) Time-domain and frequency-domain masking:

[0119] By masking audio signals in the time or frequency domain, specific sound features can be enhanced or weakened.

[0120] (5) Automatic Gain Control (AGC):

[0121] AGC (Advanced Gain Control) technology automatically adjusts the gain of the audio signal to ensure that the peak value of the output signal remains within a suitable range compared to the peak value of the input signal. The AGC algorithm monitors the peak value of the audio signal and adaptively adjusts the gain coefficient, thereby preventing signal overload or falling below noise levels.

[0122] (6) Software tools:

[0123] Using specialized software tools, such as Letasoft Sound Booster, you can increase the volume indefinitely without distortion, making it suitable for use at low volumes.

[0124] (7) Multimodal audio processing:

[0125] By combining information from the sound source and video scene analysis, multimodal audio scaling technology can be applied to improve audio quality.

[0126] (8) Professional audio software:

[0127] In professional audio production, software such as Adobe Audition and Cubase can be used to process audio gain and attenuation. These software programs offer a wealth of audio processing tools and effects.

[0128] In this technical solution, for users with hearing impairments, or in noisy environments, the current sound information acquired by the headphones can be amplified to obtain amplified sound information, which is then output. This helps the user hear the other user's voice more clearly. Through these amplification methods, the current sound information can be effectively amplified according to the user's specific needs and application scenarios to improve sound quality and listening experience.

[0129] Optionally, in some embodiments of this application, the headphones include a gain function button, and the method further includes: the headphones responding to an opening operation of the gain function button to enable the gain function;

[0130] The step of performing gain processing on the current sound information to obtain the gain-processed current sound information may include: the headphones using the gain function to perform gain processing on the current sound information to obtain the gain-processed current sound information.

[0131] In some scenarios, the current user is wearing one earphone, or a pair of earphones.

[0132] For example, if a user is currently wearing one or a pair of headphones, the user can operate the gain function button on the headphones to turn the corresponding gain function on or off according to their actual needs. Figure 4A The image shown is a schematic diagram of an embodiment of the present application, illustrating that the headphones include a gain function button.

[0133] 1) If the user is wearing headphones, for example: if the user is in a noisy environment and wants to hear the other person's voice more clearly, the user can turn on the corresponding gain function on the headphones; if the user moves from a noisy environment to a relatively quiet environment and feels that the current output sound information is too loud, the user can turn off the corresponding gain function; or if the user's hearing is impaired while wearing headphones, the user can turn on the corresponding gain function.

[0134] 2) If the user is wearing a pair of headphones, for example: regardless of whether the user's hearing is normal, if they are in a noisy environment and want to hear the other person's voice more clearly, they can turn on the gain function of both headphones; if the user moves from a noisy environment to a relatively quiet environment and feels that the current sound output is too loud, they can turn off the corresponding gain function. Another example: if the user has hearing loss in both ears and wants to hear the other person's voice more clearly, they can turn on the gain function of both headphones. Yet another example: if the user has hearing loss in one ear and normal hearing in the other ear, they can turn on the gain function of only the headphone corresponding to the ear with hearing loss, according to their needs.

[0135] In this technical solution, the user can turn on the gain function button on the headphones according to their actual needs. The headphones respond to the user's operation of turning on the gain function button, turn on the gain function, and then use the gain function to perform gain processing on the current sound information acquired by the headphones to obtain the gain-processed current sound information. Then, the gain-processed sound information is output, which helps the user hear the other party's sound information more clearly.

[0136] Optionally, in some embodiments of this application, the first display interface includes a gain function control, the smartwatch is connected to the headphones, and the method may further include: responding to an operation to enable the gain function control, enabling the gain function of the headphones; or responding to an operation to disable the gain function control, disabling the gain function of the headphones.

[0137] In some scenarios, the current user is wearing one earphone, or a pair of earphones.

[0138] For example, the headphones and smartwatch are in a connected state, meaning they need to be connected. If the current user is wearing one or both headphones, they can operate the gain function controls on the smartwatch's display interface (e.g., the smartwatch's function settings interface, or, when the headphones are connected to the smartwatch, a gain function settings interface pops up on the smartwatch) to turn the corresponding gain function of the headphones on or off, according to their actual needs. Figure 4B The image shown is a schematic diagram of the display interface of a smartwatch in an embodiment of this application, including a gain function control.

[0139] 1) If the user is currently wearing headphones, for example: if the user is in a noisy environment and wants to hear the other person's voice more clearly, the user can turn on the gain function corresponding to the headphones they are wearing on the smartwatch; if the user moves from a noisy environment to a relatively quiet environment and feels that the current output sound information is too loud, the user can turn off the gain function corresponding to the headphones they are wearing on the smartwatch; or, for example: if the user's hearing is impaired while wearing headphones, the user can turn on the corresponding gain function on the smartwatch.

[0140] 2) If the user is currently wearing a pair of headphones, for example: regardless of whether the user's hearing is normal, if they are in a noisy environment and want to hear the other person's voice more clearly, they can turn on the gain function of both headphones on the smartwatch; if the user moves from a noisy environment to a relatively quiet environment and feels that the current sound output is too loud, they can turn off the corresponding gain function on the smartwatch. For example: if the user has hearing loss in both ears and wants to hear the other person's voice more clearly, they can turn on the gain function of both headphones on the smartwatch. For example: if the user has hearing loss in one ear and normal hearing in the other ear, they can turn on the gain function of only the headphone corresponding to the ear with hearing loss on the smartwatch, according to their needs.

[0141] In this technical solution, the user can turn the headphone's gain function on or off by operating the gain function control on the smartwatch's display interface. If the gain function is turned on, the headphone can use the gain function to process the current sound information acquired by the headphone, obtain the gain-processed current sound information, and then output the gain-processed sound information. This helps the user hear the other party's voice more clearly.

[0142] Optionally, in some embodiments of this application, the first display interface includes a gain function control, the smartwatch is connected to the headphones, and the method may further include: the smartwatch responds to the activation operation of the gain function control and activates the gain function for the headphones; the smartwatch performs gain processing on the current sound information to obtain the gain-processed current sound information, which may include: the smartwatch using the activated gain function to perform gain processing on the current sound information to obtain the gain-processed current sound information;

[0143] Alternatively, the smartwatch may respond to a shutdown operation of the gain function control by disabling the gain function for the headphones.

[0144] In this technical solution, the user can enable or disable the corresponding gain function on the smartwatch by operating the gain function control on the smartwatch's display interface. If the gain function is enabled, the smartwatch can use the gain function to perform gain processing on the current audio information, obtaining the gain-processed audio information, and then send the gain-processed audio information to the headphones. The headphones output the gain-processed audio information, which helps the user hear the other party's voice more clearly.

[0145] Optionally, in some embodiments of this application, the earphone includes a gain function button, the earphone is connected to the smartwatch, and the first display interface of the smartwatch includes the gain function control of the earphone.

[0146] For example, the headphones and smartwatch are in a connected state, meaning they need to be connected. If the current user is wearing one or a pair of headphones, then the current user can operate the gain function button on the headphones, or the gain function control on the smartwatch's display interface (such as the function settings interface in the smartwatch, or the gain function settings interface that pops up in the smartwatch when the headphones are connected to the smartwatch), to turn the corresponding gain function of the headphones on or off according to their actual needs.

[0147] In this technical solution, the gain function button on the user's headphones, or the gain function control on the smartwatch's display interface, can be used to turn the corresponding gain function on or off. If the gain function is turned on, the headphones can use it to perform gain processing on the current sound information acquired by the headphones, obtaining the gain-processed sound information, and then outputting the gain-processed sound information. This helps the user hear the other party's voice more clearly.

[0148] In this embodiment, current audio information is acquired; key content information is extracted from the current communication content information corresponding to the current audio information to obtain current key content information, which is simplified information used to express the current communication content information corresponding to the current audio information; the current key content information is displayed on the first display interface of the display device. This allows the smartwatch to display the current key content information obtained by extracting key information from the current audio information when communicating with another user, thereby assisting the current user in understanding the other user's communication content. Optionally, the smartwatch can also send the current audio information to headphones, which output the current audio information to help the current user hear the other user's speech; simultaneously, the smartwatch transmits the current audio information to the built-in AI processing unit, which quickly analyzes the other user's speech content based on preset speech recognition and natural language processing algorithms, and extracts and summarizes it into key information or points; the key information or points processed by AI will be displayed on the smartwatch's display interface, allowing the user to easily view the core and key points of the other user's speech by simply raising their wrist. This method is particularly suitable when the other party speaks quickly or the content is lengthy, helping the current user maintain synchronization in communication comprehension.

[0149] like Figure 5 The diagram shown is a schematic representation of another embodiment of the speech processing method in this application, which may include:

[0150] 501. Obtain current sound information.

[0151] 502. The current communication content information corresponding to the current sound information is processed by extracting key content information to obtain current key content information, which is simplified information used to express the current communication content information corresponding to the current sound information.

[0152] 503. Display the current key content information on the first display interface of the display device.

[0153] It should be noted that, in the embodiments of this application, steps 501-503 can be referred to Figure 2 Steps 201-203 in the illustrated embodiment will not be repeated here.

[0154] 504. Upon detecting a trigger command to view historical communication content, a second display interface is displayed through the display device. The second display interface includes a first viewing option and a second viewing option, wherein the first viewing option is used to view historical communication content information, and the second viewing option is used to view historical key content information corresponding to the historical communication content information.

[0155] 505. Upon detecting a selection instruction for a target viewing option, a third display interface is displayed through the display device. The third display interface displays content information corresponding to the target viewing option, wherein the target viewing option is either the first viewing option or the second viewing option.

[0156] For example, the above-mentioned instruction to view historical communication content can be triggered by the current user performing a touch operation (e.g., pressing, double-clicking, swiping, etc.) on the historical communication content viewing control, or by the current user triggering it via voice; the above-mentioned instruction to select the target viewing option can be triggered by the current user performing a touch operation (e.g., pressing, double-clicking, swiping, etc.) on the target viewing option control, or by the current user triggering it via voice, and the specific embodiments of this application are not limited thereto.

[0157] like Figure 4C The diagram illustrates an embodiment of how a user views communication content within this application. For example, after a communication session ends, the user can view the communication content. However, when viewing the communication content, the user can choose to view the complete communication content or only the key information within it. If the user feels that the key information displayed on the smartwatch's screen is not detailed enough and wants a complete understanding of the communication, they can choose to view the complete communication content. Conversely, if the user finds the smartwatch's screen size limited and viewing the complete communication content visually inconvenient, they can also choose to view only the key information within the communication content.

[0158] In this technical solution, after each communication session, the current user can view the complete communication content (conversation, or call (e.g., voice or video call)) or key information corresponding to the communication content on their smartwatch. This helps the current user review and refer to the complete content of the communication. The current user can choose to view either the complete communication content or the key information corresponding to the communication content, providing different options to meet diverse user needs and thus improving the user experience.

[0159] Optionally, in some embodiments of this application, the communication content information in the historical communication content information that does not correspond to the voice information of the current user is detected by the smartwatch, and the communication content information in the historical communication content information that corresponds to the voice information of the current user is detected by the earphone and sent to the smartwatch.

[0160] In this technical solution, communication content information corresponding to voice information other than the current user can be detected and stored primarily through a smartwatch, while communication content information corresponding to the current user's voice information can be detected primarily through headphones and then sent to the smartwatch for storage. The headphones and smartwatch work together to improve work efficiency.

[0161] In this embodiment, current voice information is acquired; key content information is extracted from the current communication content information corresponding to the current voice information to obtain current key content information, which is simplified information used to express the current communication content information corresponding to the current voice information; the current key content information is displayed on a first display interface of the display device. This is used to display the current key content information obtained by extracting key information from the current voice information on a smartwatch when communicating with another user, thereby assisting the current user in understanding the other user's communication content. Upon detecting a trigger command to view historical communication content, a second display interface is displayed through the display device. The second display interface includes a first viewing option and a second viewing option, wherein the first viewing option is used to view historical communication content information, and the second viewing option is used to view historical key content information corresponding to the historical communication content information; upon detecting a selection command for a target viewing option, a third display interface is displayed through the display device, displaying content information corresponding to the target viewing option, which is either the first viewing option or the second viewing option. This is used so that after each communication ends, the current user can view historical communication content information, or historical key content information corresponding to the historical communication content information, on the smartwatch. This helps the current user review and refer to the complete content of this communication process.

[0162] In the above Figure 2 and Figure 5The illustrated embodiment not only enhances the user's comprehension—by using AI to summarize voice information in real time, enabling the user to quickly understand the other party's intent even if the speech is fast or the content is complex—but also improves communication efficiency: the user doesn't need to repeatedly ask or request the other party to slow down; they can keep up with the conversation by viewing key information on the smartwatch's display. It also provides a convenient user experience; the user can easily adjust and view key information corresponding to the voice message on the smartwatch's display without additional equipment. Furthermore, it allows for personalized settings; the user can adjust the simplification level, language type, and display format of key information according to their needs, making the smartwatch and headphone collaboration system more aligned with personal habits and preferences. This system design considers the user's hearing assistance needs and the ease of accessing sound information, improving the efficiency and quality of communication through intelligent technology.

[0163] like Figure 6 The diagram shown is a schematic representation of an embodiment of a smartwatch in this application. The smartwatch includes a display device 601 and further comprises:

[0164] The processing device 602 is used to acquire current sound information; extract key content information from the current communication content information corresponding to the current sound information to obtain current key content information, wherein the current key content information is simplified information used to express the current communication content information corresponding to the current sound information; and display the current key content information on the first display interface of the display device 601.

[0165] Optionally, in some embodiments of this application, the processing device 602 is further configured to, upon detecting a trigger instruction to view historical communication content, display a second display interface via the display device 601. The second display interface includes a first viewing option and a second viewing option, wherein the first viewing option is used to view historical communication content information, and the second viewing option is used to view historical key content information corresponding to the historical communication content information; upon detecting a selection instruction for a target viewing option, display a third display interface via the display device 601, wherein the third display interface displays content information corresponding to the target viewing option, and the target viewing option is either the first viewing option or the second viewing option.

[0166] Optionally, in some embodiments of this application, the processing device 602 is further configured to display a fourth display interface through the display device 601 when a simplified setting trigger command is detected. The fourth display interface includes at least one simplified option. When at least two simplified options are included, different simplified options extract key content information from the communication content information, resulting in different degrees of simplification of the key content information.

[0167] The processing device 602 is specifically used to, upon detecting a selection instruction for a target simplification option, use the simplification level corresponding to the target simplification option to extract key content information from the current communication content information corresponding to the current voice information to obtain current key content information, wherein the at least two simplification options include the target simplification option.

[0168] Optionally, in some embodiments of this application, the third display interface further includes reference communication content information and at least one reference key content information, wherein the at least one reference key content information corresponds to the at least one simplified option;

[0169] The target reference key content information is used to indicate the degree of simplification obtained by applying the target simplification option to the reference communication content information;

[0170] The at least one reference key content information includes the target reference key content information.

[0171] Optionally, in some embodiments of this application, the processing device 602 is further configured to display a fifth display interface through the display device 601 when a language setting trigger command is detected. The fifth display interface includes at least two language options, and different language options correspond to different language types for displaying the current key content information.

[0172] The processing device 602 is specifically used to display the current key content information on the first display interface of the display device 601 using the language type corresponding to the target language option when a selection instruction for the target language option is detected.

[0173] Optionally, in some embodiments of this application, the processing device 602 is specifically used to extract key content information from the current communication content information corresponding to the current sound information through an artificial intelligence processing unit to obtain current key content information.

[0174] Optionally, in some embodiments of this application, the processing device 602 is specifically used to extract key content information from the current communication content information corresponding to the current sound information by using a preset speech recognition algorithm and natural language processing algorithm in the artificial intelligence processing unit.

[0175] Optionally, in some embodiments of this application, the processing device 602 is specifically used to receive the current sound information sent by the earphone through the receiving device 603, wherein the smartwatch is connected to the earphone; or, to detect the current sound information.

[0176] Optionally, in some embodiments of this application, the first display interface includes a gain function control. The smartwatch is connected to the earphones, and the processing device 602 is further configured to respond to an operation to enable the gain function of the earphones by enabling the gain function control; or, respond to an operation to disable the gain function of the earphones by disabling the gain function control.

[0177] like Figure 7 The image shown is a schematic diagram of another embodiment of the smartwatch in this application. The following is a related illustration. Figure 7 A detailed introduction to the various components of a smartwatch:

[0178] RF circuit 710 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 780; additionally, it transmits uplink data to the base station. Typically, RF circuit 710 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 710 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0179] The memory 720 can be used to store software programs and modules. The processor 780 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0180] The input unit 730 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 731), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 780, and can also receive and execute commands sent by the processor 780. In addition, the touch panel 731 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 731, the input unit 730 may also include other input devices 732. Specifically, other input devices 732 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0181] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 740 may include a display panel 741, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Further, a touch panel 731 may cover the display panel 741. When the touch panel 731 detects a touch operation on or near it, it transmits the information to the processor 780 to determine the type of touch event. Subsequently, the processor 780 provides corresponding visual output on the display panel 741 based on the type of touch event. Although in Figure 7 In this embodiment, the touch panel 731 and the display panel 741 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 731 and the display panel 741 can be integrated to realize the input and output functions of the mobile phone.

[0182] The mobile phone may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 741 according to the ambient light level, and the proximity sensor can turn off the display panel 741 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0183] Audio circuit 760, speaker 761, and microphone 762 provide an audio interface between the user and the mobile phone. Audio circuit 760 converts received audio data into electrical signals and transmits them to speaker 761, where speaker 761 converts them into sound signals for output. On the other hand, microphone 762 converts collected sound signals into electrical signals, which are received by audio circuit 760, converted into audio data, and then processed by processor 780 before being transmitted via RF circuit 710 to, for example, another mobile phone, or the audio data can be output to memory 720 for further processing.

[0184] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 770, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 7 The Wi-Fi module 770 is shown, but it is understood that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0185] The processor 780 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 720, and calls data stored in the memory 720 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 780.

[0186] The mobile phone also includes a power supply 790 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 780 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0187] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0188] In this embodiment, the processor 780 is configured to acquire current sound information; extract key content information from the current communication content information corresponding to the current sound information to obtain current key content information, wherein the current key content information is simplified information used to express the current communication content information corresponding to the current sound information; and display the current key content information on the first display interface of the display unit 740.

[0189] Optionally, in some embodiments of this application, the processor 780 is further configured to, upon detecting a trigger instruction to view historical communication content, display a second display interface through the display unit 740. The second display interface includes a first viewing option and a second viewing option, wherein the first viewing option is used to view historical communication content information, and the second viewing option is used to view historical key content information corresponding to the historical communication content information; upon detecting a selection instruction for a target viewing option, display a third display interface through the display unit 740, wherein the third display interface displays content information corresponding to the target viewing option, and the target viewing option is either the first viewing option or the second viewing option.

[0190] Optionally, in some embodiments of this application, the processor 780 is further configured to display a fourth display interface through the display unit 740 when a simplified setting trigger instruction is detected. The fourth display interface includes at least one simplified option. When at least two simplified options are included, different simplified options extract key content information from the communication content information, resulting in different degrees of simplification of the key content information.

[0191] The processor 780 is specifically configured to, upon detecting a selection instruction for a target simplification option, use the simplification level corresponding to the target simplification option to extract key content information from the current communication content information corresponding to the current voice information to obtain current key content information, wherein the at least two simplification options include the target simplification option.

[0192] Optionally, in some embodiments of this application, the third display interface further includes reference communication content information and at least one reference key content information, wherein the at least one reference key content information corresponds to the at least one simplified option;

[0193] The target reference key content information is used to indicate the degree of simplification obtained by applying the target simplification option to the reference communication content information;

[0194] The at least one reference key content information includes the target reference key content information.

[0195] Optionally, in some embodiments of this application, the processor 780 is further configured to display a fifth display interface through the display unit 740 when a language setting trigger instruction is detected. The fifth display interface includes at least two language options, and different language options correspond to different language types for displaying the current key content information.

[0196] The processor 780 is specifically configured to, upon detecting a selection instruction for a target language option, display the current key content information on the first display interface of the display unit 740 using the language type corresponding to the target language option.

[0197] Optionally, in some embodiments of this application, the processor 780 is specifically used to extract key content information from the current communication content information corresponding to the current sound information through an artificial intelligence processing unit to obtain current key content information.

[0198] Optionally, in some embodiments of this application, the processor 780 is specifically used to extract key content information from the current communication content information corresponding to the current sound information by using a preset speech recognition algorithm and natural language processing algorithm in the artificial intelligence processing unit.

[0199] Optionally, in some embodiments of this application, the processor 780 is specifically configured to receive the current sound information sent by the earphone through the RF circuit 710, wherein the smartwatch is connected to the earphone; or, to detect the current sound information.

[0200] Optionally, in some embodiments of this application, the first display interface includes a gain function control. The smartwatch is connected to the earphones, and the processor 780 is further configured to respond to an operation to enable the gain function control to enable the gain function of the earphones; or, respond to an operation to disable the gain function control to disable the gain function of the earphones.

[0201] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0202] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0204] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0205] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0206] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0207] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A voice processing method, characterized by, The method is applied to a smart watch provided with a display device, and comprises the following steps: obtaining current sound information; extracting key content information from current communication content information corresponding to the current sound information to obtain current key content information, the current key content information being simplified information for expressing the current communication content information corresponding to the current sound information; displaying the current key content information on a first display interface of the display device.

2. The method of claim 1, wherein, The method further comprises: in the case of detecting a historical communication content viewing trigger instruction, displaying a second display interface through the display device, the second display interface comprising a first viewing option and a second viewing option, wherein the first viewing option is used to view historical communication content information, and the second viewing option is used to view historical key content information corresponding to the historical communication content information; in the case of detecting a selection instruction for a target viewing option, displaying a third display interface through the display device, the third display interface displaying content information corresponding to the target viewing option, the target viewing option being the first viewing option or the second viewing option.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: in the case of detecting a simplification setting trigger instruction, displaying a fourth display interface through the display device, the fourth display interface comprising at least one simplification option, wherein in the case of comprising at least two simplification options, different simplification options perform key content information extraction on communication content information to obtain key content information with different degrees of simplification; the method further comprises: in the case of detecting a selection instruction for a target simplification option, using a degree of simplification corresponding to the target simplification option to extract key content information from the current communication content information corresponding to the current sound information to obtain current key content information, the at least two simplification options comprising the target simplification option.

4. The method of claim 3, wherein, The third display interface further comprises reference communication content information and at least one reference key content information, the at least one reference key content information corresponding to the at least one simplification option; target reference key content information is used to indicate reference key content information obtained by simplifying the reference communication content information using a degree of simplification corresponding to the target simplification option; wherein the at least one reference key content information comprises the target reference key content information.

5. The method according to claim 1 or 2, characterized in that, The method further comprises: in the case of detecting a language setting trigger instruction, displaying a fifth display interface through the display device, the fifth display interface comprising at least two language options, different language options corresponding to different language types of the current key content information; the method further comprises: in the case of detecting a selection instruction for a target language option, displaying the current key content information on the first display interface of the display device using a language type corresponding to the target language option.

6. The method of claim 1 or 2, wherein, The current communication content information corresponding to the current sound information is subjected to key content information extraction to obtain current key content information, including: The current communication content information corresponding to the current sound information is subjected to key content information extraction by the artificial intelligence processing unit to obtain current key content information.

7. The method of claim 6, wherein, The current communication content information corresponding to the current sound information is subjected to key content information extraction by the artificial intelligence processing unit to obtain current key content information, including: The current communication content information corresponding to the current sound information is subjected to key content information extraction by the artificial intelligence processing unit to obtain current key content information.

8. The method of claim 1 or 2, wherein, The current communication content information corresponding to the current sound information is subjected to key content information extraction by the artificial intelligence processing unit to obtain current key content information. The current sound information is obtained, including: The current sound information sent by the earphone is received, wherein the smart watch is connected with the earphone; or, 9. The method of claim 1 or 2, wherein, The current sound information is detected. The first display interface includes a gain function control, the smart watch is connected with the earphone, and the method further includes: In response to an opening operation on the gain function control, the gain function for the earphone is opened; or 10. A smart watch, characterized by In response to a closing operation on the gain function control, the gain function for the earphone is closed. The smart watch is provided with a display device, and the smart watch further includes:

11. A computer readable storage medium having stored thereon a computer program, characterized in that, A processing device is configured to obtain current sound information, subject the current communication content information corresponding to the current sound information to key content information extraction to obtain current key content information, the current key content information being simplified information for expressing the current communication content information corresponding to the current sound information, and display the current key content information on a first display interface of the display device.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the method of any one of claims 1-9. The computer program is executed by a processor to implement the method of any one of claims 1-9.

Images