A method, terminal and medium for realizing voice statistics function

Abstract

The embodiments of the present application provide a method, terminal and medium for implementing a voice statistics function. The method is as follows: in response to the voice statistics function being in an on state, the application processor of the terminal displays the voice statistics in the voice display area of ​​the display interface; in response to the voice statistics function being continuously in an on state, if a user's voice input is received, the updated voice statistics are displayed in the voice display area of ​​the display interface; the updated voice statistics are the current voice statistics obtained by updating the voice volume based on the voice input. That is, when the user speaks, the terminal device recognizes that the user is speaking, and adjusts the display value of the voice display area based on the user's speaking volume, thereby implementing the voice statistics function in the terminal display interface. When the display interface is a lock screen interface, the lock screen interface can also display the user's voice statistics, thereby improving the fun of the lock screen interface.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method, terminal and medium for implementing a voice statistics function. Background Art

[0002] As living standards improve, people are paying more and more attention to the health of themselves and their families. The amount of user speech can be used together with sleep data, stress data, body temperature and body fat data to measure the user's health information. Therefore, it is extremely important to pay attention to the amount of user speech. The amount of user speech, also known as the user's voice statistics, refers to the total amount of speech that the user speaks in a preset time period such as a day or a month.

[0003] The diversification of terminals has promoted the diversification of terminal functions. Nowadays, a terminal can integrate communication, photography, video playback, games and other functions, which greatly meets user needs. However, the terminal lock screen interface only displays the user's exercise steps, which is not interesting enough. Based on this, associating the terminal with the user's voice statistics can not only increase the fun of the lock screen interface, but also provide reference data for measuring the user's health information. Summary of the invention

[0004] The embodiments of the present application provide a method, terminal and medium for implementing a voice statistics function, which are used to integrate a voice statistics function in a terminal and display the user's voice statistics on the terminal lock screen interface, which can not only increase the fun of the lock screen interface, but also provide reference data for measuring the user's health information.

[0005] In the first aspect, an embodiment of the present application provides a method for implementing a voice statistics function. In response to the voice statistics function being in an on state, the application processor of the terminal displays the voice statistics in the voice display area of ​​the display interface; in response to the voice statistics function being continuously in an on state, if a user's voice input is received, the updated voice statistics are displayed in the voice display area of ​​the display interface; the updated voice statistics are the current voice statistics obtained by updating the voice volume based on the voice input. In other words, when the user speaks, the terminal device recognizes that the user is speaking, and adjusts the display value of the voice display area based on the user's speaking volume, thereby implementing the voice statistics function in the terminal display interface. When the display interface is a lock screen interface, the lock screen interface can also display the user's voice statistics, thereby improving the fun of the lock screen interface and providing reference data for measuring the user's health information.

[0006] In a possible implementation, the terminal further includes an audio codec and a smart sensor hub Sensorhub. After receiving the user's voice input, the audio codec converts the user's voice input into an audio data stream; the Sensorhub obtains the audio data stream from the audio codec, and based on the audio data stream, obtains the voice volume of the voice input; based on the voice volume of the voice input, obtains the updated voice statistics; and reports the updated voice statistics to the application processor to display the updated voice statistics in the voice display area. Therefore, the processing of the voice statistics is deployed in the Sensorhub, so that the voice statistics function runs in a low power consumption mode.

[0007] Optionally, when the audio data stream is a pulse code modulation PCM stream, the Sensorhub obtains the PCM stream from the audio codec; performs noise reduction processing on the PCM stream; and obtains the voice volume of the voice input using the PCM stream after the noise reduction processing. Thus, through the noise reduction processing, the influence of noise on the obtained voice volume is avoided, and the acquisition accuracy is improved.

[0008] In another possible implementation, the SensorHub reports to the application processor once every first time interval in the screen-on mode, and reports to the application processor once every second time interval in the screen-off mode; the second time interval is greater than the first time interval. Thus, in the screen-off mode, the power consumption of the terminal is reduced by reducing the number of times the application processor is woken up.

[0009] In another possible implementation, after receiving voice inputs from multiple users, the target user is matched from the voice inputs from multiple users based on the obtained identification information of the target user; and the updated voice statistics of the target user are displayed in the voice display area. For example, the terminal receives voice inputs from user A, user B, and user C at the same time. The terminal matches the target user from the received multiple users based on the identification information of the target user. For example, if the target user is A, only the voice statistics of target user A are displayed in the voice display area of ​​the terminal display interface, so that the voice statistics function is more targeted.

[0010] In another possible implementation, voice statistics for N consecutive cycles are obtained; N is an integer greater than 1; and the voice statistics for N consecutive cycles and the change trend are displayed in the voice display area of ​​the display interface. Thus, the user's health information can be measured by the change trend of the voice statistics for N consecutive cycles.

[0011] In another possible implementation, the application processor displays a setting interface in response to the voice statistics function being in an on state; the setting interface includes a first function selection item of the voice statistics function, and the first function selection item is used to turn on or off the voice statistics function of the terminal. In response to the on operation of turning on the first function selection item of the setting interface, the voice statistics function is turned on.

[0012] Optionally, the setting interface also includes a second function selection item of the voice statistics function, and the second function selection item is used to determine the display rules of the voice statistics function. In response to a trigger operation on the second function selection item of the setting interface, the display rules of the voice statistics function are determined.

[0013] Optionally, the display rule includes one of a sentence-based display rule, a word-based display rule, and a phrase-based display rule.

[0014] In a second aspect, the present application discloses a terminal, including a processor and a memory; the memory is used to store computer execution commands; the processor is used to execute the computer execution commands stored in the memory, so that the processor executes the implementation method of the voice statistics function as in the first aspect.

[0015] In a third aspect, the present application discloses a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, a method for implementing the speech statistics function as in the first aspect is implemented.

[0016] In a fourth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method for implementing the speech statistics function in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A display schematic diagram of a terminal display interface provided in an embodiment of the present application;

[0018] Figure 2 A display schematic diagram of another terminal display interface provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of turning on a call counting function provided in an embodiment of the present application;

[0020] Figure 4 A display schematic diagram of another terminal display interface provided in an embodiment of the present application;

[0021] Figure 5 A distribution diagram of display cards;

[0022] Figure 6A display schematic diagram of a terminal display interface provided in an embodiment of the present application;

[0023] Figure 7 A flow chart of a method for implementing a voice statistics function provided in an embodiment of the present application;

[0024] Figure 8 A schematic diagram of a multi-user conversation provided in an embodiment of the present application;

[0025] Fig. 9 This is the hardware architecture diagram of the terminal;

[0026] Fig.10 A schematic diagram of a terminal software architecture diagram provided in an embodiment of the present application;

[0027] Fig.11A It is a software block diagram for ADSP implementation;

[0028] Fig. 11B This is another software block diagram for ADSP implementation;

[0029] Fig. 12A A complete implementation structure block diagram of a specific voice statistics function is provided for the embodiment of the present application;

[0030] Fig. 12B Another complete implementation structure block diagram of a specific voice statistics function is provided for the embodiment of the present application;

[0031] Fig. 12C A functional structural block diagram of a call counter provided in an embodiment of the present application;

[0032] Fig.13A An interactive diagram of a method for implementing a voice statistics function provided in an embodiment of the present application;

[0033] Fig. 13B An interactive diagram of an implementation method for disabling the voice statistics function provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The terms "first", "second", "third", etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0035] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0036] The embodiments of the present application are applied to a terminal with a display function. The terminal can be in various forms, for example, a mobile phone, a tablet computer, a computer with a wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a vehicle-mounted terminal device, a wearable terminal device, etc. The terminal can be a fixed terminal or a mobile terminal.

[0037] Currently, the terminal lock screen interface only displays the user's exercise steps, and the displayed content is simple and not interesting enough.

[0038] In view of the above problems, an embodiment of the present application provides a method for implementing a voice statistics function. In response to the voice statistics function being in an on state, the application processor of the terminal displays the voice statistics in the voice display area of ​​the display interface; in response to the voice statistics function being continuously in an on state, if a user's voice input is received, the updated voice statistics are displayed in the voice display area of ​​the display interface; the updated voice statistics are the current voice statistics obtained by updating the voice volume based on the voice input. That is to say, when the user speaks, the terminal device recognizes that the user is speaking, and adjusts the display value of the voice display area based on the user's speaking volume, thereby implementing the voice statistics function in the terminal display interface. When the display interface is a lock screen interface, the lock screen interface can also display the user's voice statistics. Therefore, the method for implementing the voice statistics function provided in the embodiment of the present application can not only increase the fun of the lock screen interface, but also provide reference data for measuring the user's health information.

[0039] The voice statistics function is also called the call counting function. It is a function that counts the total amount of voice of the user from the preset initial time to a specific time. The total amount of voice is also called voice statistics, or the total number of utterances. For example, the user has spoken a total of 80 sentences from the preset initial time to the first time, totaling 1376 characters, involving 240 words. The total amount of voice is 80 sentences, which can also be expressed as 1376 characters, or 240 words.

[0040] Combine the following Figure 1 to Figure 5 As shown, the implementation method of the voice statistics function provided in the embodiment of the present application is described. Figure 1-2 The voice display area is directly located on the display interface, and the display interface is explained using the lock screen interface as an example.

[0041] See also Figure 1 , which is a display schematic diagram of a terminal display interface provided in an embodiment of the present application. Figure 1The display interface of the terminal 100 shown in FIG. 1 is a lock screen interface 101. The lock screen interface includes a step count display area 103 and a photo taking mark 104. The step count display area 103 displays the user's exercise step count statistics result and the exercise step count statistics mark.

[0042] When the user turns on the call counting function, the voice display area of ​​the lock screen interface 101 displays the voice statistics.

[0043] See also Figure 2 , which is a display schematic diagram of another terminal display interface provided by an embodiment of the present application. In response to the user turning on the call counting function, the lock screen interface 101 also includes a voice display area 102, and the voice display area 102 displays voice statistics and voice statistics identifiers.

[0044] The speech statistics displayed in the speech display area 102 are the total amount of speech from the preset start time to a specific time. The specific time may be the display time of the display interface, for example, Figure 2 In the interface shown, a specific time is the time when the lock screen interface 101 is displayed. The value displayed in the voice display area 102 is the voice statistics from the preset initial time to the display time of the lock screen interface 101. In addition, a specific time can also be a time preset by the user, which is not specifically limited in this application.

[0045] The preset start time is a time set by those skilled in the art as needed. For example, to understand the voice statistics of a day, the preset start time can be set to the start time of the day "00:00:00". Exemplarily, the preset start time is 00:00:00, and the display time of the lock screen interface 101 is 8:00:00, that is, the value 1376 displayed in the voice display area 102 is the user's voice statistics in a day up to the current time. Exemplarily, Figure 2 The voice statistics displayed in the voice display area of ​​the display interface shown are 1376 words.

[0046] In the embodiment of the present application, the call counting function can be enabled in a variety of ways.

[0047] See also Figure 3 , which is a schematic diagram of turning on the call counting function provided in an embodiment of the present application. Figure 2 The speaking volume display function in is the voice statistics function.

[0048] like Figure 3 As shown in (a) of the figure, the user opens the "Settings" application and enters Figure 3 In the setting interface shown in (b), the user can find the first function selection item in the setting interface, and the first function selection item is displayed as closed, indicating that the speaking volume display function is closed. Figure 3There is no display card on the desktop interface corresponding to (a), and there is no voice display area on the lock screen interface.

[0049] The user clicks the expansion button corresponding to the "Speaking Volume Display" function to enter Figure 3 The setting interface for setting the speaking amount display function shown in (c) of FIG. In this setting interface, the user performs settings.

[0050] in, Figure 3 (c) in the figure indicates that the first function item is in the off state. The user performs an on operation to turn on the speech volume display function. The on operation can be sliding from the side showing "off" to the side showing "on". The on operation can also be other triggering operations, such as double-clicking / singling, so that the terminal 100 turns on the speech counting function.

[0051] Furthermore, after the call counting function is turned on, the terminal's setting interface also includes a second function selection item. The second function selection item is used to select a display rule for the current speaking volume. The display rule includes but is not limited to a phrase-based display rule, a word-based display rule, and a sentence-based display rule. In this way, the user can adjust the display content of the voice display area as needed, which provides a better user experience.

[0052] Figure 3 (d) in the figure indicates that the first function item is turned on, and the setting interface displays the second function selection item. The user can select a display rule in the selection button of the second function selection item. Specifically, the display rule includes a character-based display rule, a word-based display rule, and a sentence-based display rule. Those skilled in the art can select it as needed. Figure 3 (d) in the figure shows the user selecting a word-based display rule.

[0053] Need to explain, Figure 3 The implementation method of the display rules in the description is only for illustration purposes. For example, the “speech volume display function” is located in the “Health” application, and users can open the “Health” application and set it there.

[0054] The embodiments of the present application do not specifically limit the setting rules. For example, the setting rules may be performed before the “speaking amount display function” is turned on, or the setting rules may be performed while the “speaking amount display function” is in operation.

[0055] See also Figure 4, which is a display schematic diagram of another terminal display interface provided by an embodiment of the present application. When the call counting function of the terminal 100 is continuously in the on state, for example, from 8:00 to 8:15, the call counting function of the terminal 100 is continuously in the on state. When the call counting function is turned on, the current statistics displayed in the voice display area 102 at 8:00 is 1376 words. If the terminal 100 receives the user's voice input at 8:05, the voice volume of the voice input is 10 words, and the current statistics displayed at 8:05 is 1386 words. If the terminal 100 receives the user's voice input at 8:10, the voice volume of the voice input is 5 words, and the current statistics displayed at 8:10 is 1391 words. If the terminal 100 receives the user's voice input at 8:15 as 9 words, the current statistics displayed at 8:15 is 1400 words.

[0056] Thus, in response to the voice statistics function being in the on state, the application processor of the terminal of the embodiment of the present application displays the voice statistics in the voice display area of ​​the display interface; in response to the voice statistics function being continuously in the on state, if the user's voice input is received, the updated voice statistics are displayed in the voice display area of ​​the display interface. Thus, the voice statistics function is implemented in the terminal display interface. When the display interface is a lock screen interface, the lock screen interface can also display the user's voice statistics, thereby improving the interest of the lock screen interface.

[0057] The embodiments of the present application do not specifically limit the display method of the voice display area. For example, the voice display area can be displayed on the lock screen interface or on a non-lock screen interface. The voice display area can be a partial area of ​​the display interface or the entire area of ​​the display interface. The voice display area can display voice statistics and user information related to the voice statistics.

[0058] The embodiment of the present application does not specifically limit the display position of the voice display area. The voice display area can be displayed at any position on the lock screen interface, such as Figure 1 and Figure 2 The display area can be displayed in the following manner: Figure 1 and Figure 2 The display card is directly located on the display interface. The display card is a small tool located on the display interface. Figure 5 , this figure is a distribution diagram of display cards.

[0059] Depend on Figure 5As can be seen, the display card 302 is located on the terminal desktop interface 300. The desktop interface 300 displays a variety of application icons, such as a "file management" application icon, an "email" application icon, a "music" application icon, etc. The user triggers the application icon to perform the application corresponding to the application icon.

[0060] The desktop display card 302 is usually larger than the icons on the desktop interface. The display card 302 can display the voice display area 102, which displays the user's total speech volume today and the average speech volume 301 of this week.

[0061] The embodiment of the present application does not specifically limit the position of the display card 302 , for example, it may be above the interface, or may be located at the end or below the interface.

[0062] In another example, the terminal display interface can display the voice statistics for N consecutive days and the change trend. Figure 6 , which is a display schematic diagram of a terminal display interface provided in an embodiment of the present application. The display interface displays a speaking volume display function.

[0063] Figure 6 The display shows the statistical results of speech volume in the past half month (2023.11.25~2023.12.10) and the trend of change. The trend of change is displayed on the display interface through a bar graph. Therefore, users can provide reference data for measuring the user's health status based on the trend of the statistical results of speech volume in the past half month, combined with sleep data, stress data, body temperature and body fat data, etc.

[0064] also, Figure 6 It also shows the comparison between today's speech volume and yesterday's speech volume. For example, yesterday's speech volume was 13,500 words, and today's speech volume was 1,376 words, which means today's speech volume was less than yesterday. Figure 6 It also shows the average speech volume in the past 15 days. For example, the average speech volume in the past 15 days is 12,000 words, and the speech volume today is 1,376 words.

[0065] It should be noted that the speech statistics for N consecutive days provided in the embodiment of the present application can be adjusted as needed. For example, N consecutive days including today can be displayed, or any N consecutive days can be selected for display.

[0066] See also Figure 7 , which is a flow chart of a method for implementing a voice statistics function provided in an embodiment of the present application. The method includes:

[0067] S11: The terminal 100 receives the user's voice information.

[0068] Voice information refers to the user's voice input information, also known as sound information, which is used to describe the user's voice tone, volume, sound frequency, and specific content of the speech. When the user speaks to the terminal 100, the terminal 100 receives the voice information.

[0069] In an embodiment of the present application, the terminal 100 includes a microphone. A microphone, also called a "microphone" or a "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can input the sound signal into the microphone by speaking close to the microphone. The terminal 100 can be provided with at least one microphone. In other embodiments, the terminal 100 can be provided with two microphones, which can not only collect sound signals but also realize a noise reduction function. In other embodiments, the terminal 100 can also be provided with three, four or more microphones to collect sound signals, reduce noise, and identify the source of sound, etc.

[0070] S12: The terminal 100 converts the user's voice information into an audio data stream.

[0071] The audio data stream is a digital voice information, which is composed of a series of binary values. In the embodiment of the present application, the terminal 100 directly obtains the voice quantity corresponding to the voice information according to the audio data stream.

[0072] The voice information received by the terminal 100 is analog information, which cannot be directly processed digitally, and needs to be converted into digital information. In the embodiment of the present application, the terminal 100 converts the voice information of the received electrical signal into digital voice information, that is, an audio data stream.

[0073] In one example, the terminal 100 includes an audio hardware encoder, which is an encoding device that converts simulated voice information into an audio data stream through hardware encoding. Specifically, the audio hardware encoder is an audio hardware codec. The audio hardware codec is a hardware-based encoder and decoder that converts simulated voice information into an audio data stream through hardware encoding, and converts the audio data stream into simulated voice information through hardware decoding. In an embodiment of the present application, the microphone of the terminal 100 sends the voice information of the electrical signal to the audio hardware codec, and the audio hardware codec encodes the voice information to obtain an audio data stream.

[0074] The present application does not specifically limit the audio hardware codec, for example, it can be a pulse code modulation (Pulse Code Modulation, PCM) codec, in which case the audio data stream is a PCM code stream.

[0075] In another example, the terminal 100 includes an audio software decoder. The audio software decoder is a software program that converts analog information into digital voice information by encoding. Specifically, the microphone sends the voice information of the electrical signal to the audio software encoder, and the audio software encoder is used to encode the voice information to obtain an audio data stream.

[0076] S13: The terminal 100 identifies the audio data stream and obtains the speech volume corresponding to the audio data stream.

[0077] The speech volume refers to the amount of speech, such as how many sentences, words, or characters are spoken. In the embodiment of the present application, the speech volume is the speech volume obtained based on the audio data stream. The audio data stream is composed of a series of binary values, and the speech volume can be obtained using the audio data stream.

[0078] In an embodiment of the present application, the terminal stores three types of voice volume measurement units, namely, character-based measurement units, word-based measurement units and sentence-based measurement units. The terminal can upload voice volumes or voice statistics of different measurement units as needed.

[0079] In one example, the terminal 100 includes software drivers such as an audio framework, an audio virtual sensor, and a talker. The audio framework is connected to the audio hardware codec and communicates with the audio virtual sensor, which communicates with the talker. After the audio framework obtains the audio data stream from the audio hardware encoder, it directly sends the audio data stream to the audio virtual sensor, which sends the audio data stream to the talker. The talker identifies the audio data stream and obtains the voice volume corresponding to the audio data stream.

[0080] In another example, the audio data stream obtained by the audio framework is an original audio data stream directly converted from analog information, which may contain too much noise, affecting the accuracy of identifying the audio data stream. After the audio framework obtains the original audio data stream, it can perform noise reduction on the original audio data stream, and send the noise-reduced audio data stream to the audio virtual sensor, which is then sent to the call counter. The call counter identifies the audio data stream and obtains the number of voices corresponding to the audio data stream.

[0081] In one example, after the call counter recognizes the audio data stream, it can directly obtain the voice volume corresponding to the audio data stream.

[0082] In another example, after the talker obtains the audio data stream, it first preprocesses the audio data stream, identifies the preprocessed audio data stream, and obtains the voice volume corresponding to the audio data stream. Preprocessing refers to clearing abnormal data in the audio data stream and correcting erroneous data, thereby improving the accuracy of identifying the audio data stream. For example, there are some values ​​between "0" and "1" in the PCM code stream. These values ​​belong to abnormal data that are converted into existence. If they are not processed, the talker may not be able to recognize them, resulting in errors in obtaining the voice volume. The embodiments of the present application do not specifically limit the technical implementation methods of preprocessing. For example, the preprocessing method may include data cleaning, data integration, data transformation, or data reduction.

[0083] In another example, the audio data stream obtained by the call counter may be a compressed audio data stream, the purpose of which is to reduce the channel resources occupied during the transmission of the audio data stream and improve the transmission speed. After the call counter obtains the audio data stream, it first decompresses the audio data stream, and then pre-processes and recognizes the decompressed audio data stream.

[0084] In addition, the user sometimes needs to know the total amount of voice of the target user. In this case, the call counting function of the terminal 100 only counts the total amount of voice of the target user.

[0085] In one example, if the terminal 100 receives only one user's voice input, the terminal 100 pre-stores the identification information of the target user. The identification information is used to identify whether the voice information received by the terminal 100 belongs to the voice information sent by the target. For example, the identification information may be the target user's voice frequency, timbre, and pitch.

[0086] When receiving the audio data stream, the terminal 100 simultaneously obtains information related to the identification information in the voice information. The terminal 100 determines whether the user is a target user based on the information related to the identification information in the received voice information of the user. Specifically, the terminal 100 determines whether the acquired sound frequency is within the pre-stored frequency range, and whether the characteristic similarity between the timbre and the timbre of the target user is greater than a preset threshold, and whether the acquired user's pitch is within the preset pitch range of the target user. If the user is determined to be a target user, the acquired voice volume is 10 words based on the audio data stream of the target user. Otherwise, the user is determined to be a non-target user, and the voice volume corresponding to the audio data stream is 0.

[0087] Furthermore, the audio data stream carries information related to the identification information, and the terminal 100 separates the information related to the identification information from the audio data stream, and uses the separated information related to the identification information to determine whether the user is the target user, and if so, determines that the audio data stream is the audio data stream of the target user. If not, determines that the audio data stream is not the audio data stream of the target user.

[0088] In another example, the terminal 100 may receive voice information of multiple users. Figure 8 As shown, this figure is a schematic diagram of a multi-user conversation provided by an embodiment of the present application. Among them, user A, user B, and user C are all within the sound information receiving range 301 of the microphone of terminal 100. User A, user B and user C speak at the same time at a specific moment. The embodiment of the present application pre-stores the identification information of the target user. Terminal 100 first identifies the target user from multiple users based on the identification information of the target user. Terminal 100 can obtain the audio data stream of the target user from the audio data streams of multiple users based on the identification information. For example, for Figure 3 As shown, the identified target user is user A. The call counter only counts the voice volume of user A.

[0089] S14: Acquire speech statistics and display them on a display interface.

[0090] In the embodiment of the present application, the voice statistics refers to the sum of the voice volume obtained in step S13 and the voice statistics before the voice information is received in step S11. The total amount of voice before the voice information is received in step S11 is called the first value, and the display time corresponding to the first value when it is displayed in the voice display area 102 of the display interface is the first time. The total amount of voice obtained after statistics is the second value, and the display time of the second value in the voice display area 102 of the display interface is the second time.

[0091] In another example, the terminal 100 presets the start time and the end time. The start time refers to the time when the total amount of voice statistics starts, and the end time refers to the time when the total amount of voice statistics ends. Determine whether the first time is the end time. If so, set the first value to 0, add the voice volume to 0, and obtain the second value. If the first time is less than the end time, the first value is the total amount of voice from the start time to the first time. Exemplary description: Assume that the start time is 2023.10.11 00:00:00, the first time is 2023.10.11 8:00:00, the end time is 2023.10.12 00:00:00, and the first value is 1376 words. Since the first time is before the end time, the second value is the voice volume + 1376 words. If the first time is 2023.10.12 00:00:00, the first time is the end time, at this time the first value is 0, and the second value is the value corresponding to the voice volume.

[0092] In another example, the call counter obtains voice statistics, for example, counts the voice statistics for one day and stores them. The stored voice statistics are also added with a preset identifier. The preset identifier corresponds to the current time. For example, if the current time is 2023.10.12, the preset identifier is the identifier corresponding to 2023.10.12.

[0093] When the terminal needs to display the voice statistics for N consecutive days, the voice counter obtains the voice statistics for N consecutive days from the storage area for pre-stored voice statistics for one day, and sends the voice statistics for N consecutive days to the terminal for display.

[0094] In another example, the call counter sends a display mode to the terminal display interface, such as a bar graph display, and the display interface displays the voice statistics for N consecutive days according to the display mode sent by the call counter.

[0095] In addition, the speech counter can also calculate the relationship between today's speech volume and the average speech statistics of N consecutive days, and display it on the display interface.

[0096] There are many ways to implement display on the display interface.

[0097] In a possible implementation, the talker of the terminal 100 includes a speaking volume reporting function, which is used to report the acquired second value to the display interface for display. Specifically, the talker sends the display value to the Motion hardware abstraction layer of the hardware abstraction layer through the Qualcomm platform message interface, and the Motion hardware abstraction layer reports the second value to the speaking volume display application through the Motion gesture service AIDL interface and the Motion gesture system application framework, so as to display it on the display interface. Among them, the speaking volume display application is used to implement the speaking volume display function.

[0098] In the embodiment of the present application, the display value can be reported to the speech volume display application in a variety of ways.

[0099] In one example, the phone counter reports the displayed value to the display interface in real time for display.

[0100] In another example, in order to reduce the number of wake-up times of the display interface of the terminal 100 and reduce power consumption, in the screen-on mode, the call counter reports the acquired value to the display interface in real time for display. In the screen-off mode, the call counter reports the acquired value once every preset time interval. For example, the call counter obtains the last reporting time, and uses the last reporting time plus the interval preset time interval to obtain the current reporting time of the second value reported this time.

[0101] The embodiment of the present application does not specifically limit the preset time length, for example, 30 minutes, 40 minutes, 1 hour, etc.

[0102] In addition, the speech statistics displayed in the speech display area can be displayed periodically. Figure 4It is implemented within a call counting cycle T, which is the time length of the preset end time-the preset start time. That is, the moment when the terminal receives the voice information is before the preset start time and the preset end time. For the convenience of representation, the moment when the terminal receives the voice information is called the second moment.

[0103] If the second time is not between the preset start time and the preset end time, adjust the preset start time and the preset end time so that the second time is between the preset start time and the preset end time, and re-execute Figure 4 .

[0104] It can be seen that the embodiment of the present application converts the acquired voice information into an audio data stream, acquires the voice volume according to the audio data stream, counts the total voice volume, and then displays the total voice volume obtained by counting on the display interface. Thus, the voice statistics function is realized in the terminal 100.

[0105] The following is combined with Fig. 9 ~Attached Fig. 11B , which introduces in detail the architecture diagram of the terminal 100 implementation interface.

[0106] See also Fig. 9 , which is a hardware architecture diagram of the terminal 100. The terminal 100 includes a processor 110, a microphone 170C, a display screen 194, an audio hardware codec 145, an accelerometer sensor 180E, and the like.

[0107] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0108] The processor 110 includes an application processor (AP), which is used to display the total amount of voice reported by the kernel layer on the terminal display interface. Specifically, it is displayed in the voice display area of ​​the terminal display interface. The AP is also used to display the total number of movement steps reported by the kernel layer on the terminal display interface.

[0109] In some examples, processor 110 includes an inter-integrated circuit sound (I2S) interface and a universal asynchronous receiver / transmitter (UART) interface. The I2S interface can be used for audio communication.

[0110] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0111] Microphone 170C. Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by bringing their mouth close to the microphone 170C to input the sound signal into the microphone 170C. At least one microphone 170C may be provided in the embodiment of the present application. In other embodiments, the terminal may be provided with two microphones 170C, which, in addition to collecting sound signals, may also implement a noise reduction function. In other embodiments, the terminal may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, and identify the source of the sound, etc.

[0112] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in various directions (generally three axes), and is used to identify the terminal posture, and is applied to applications such as horizontal and vertical screen switching, and step counting.

[0113] See also Fig.10 , which is a schematic diagram of the terminal software architecture diagram provided in an embodiment of the present application.

[0114] The software layer of the terminal 100 includes a software system of the terminal 100 that can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal 100.

[0115] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime, the system library, the hardware abstraction layer, and the kernel layer.

[0116] The application layer may include a series of application packages, for example, a speech volume display function and a step counting function.

[0117] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework includes a display system application framework and a display system Android Interface Definition Language (AIDL) interface. The display system AIDL interface is used to obtain the total amount of voice from the hardware abstraction layer, or the total number of motion steps, and upload the total amount of voice or the total number of motion steps to the display system application framework, and the display system application framework uploads the total amount of voice or the total number of motion steps to the application layer for display. Exemplarily, the display system application framework is a device management framework. The display system AIDL interface is a Motion AIDL interface.

[0118] The hardware abstraction layer may include multiple library modules, and the Android system may load corresponding library modules for the device hardware, thereby realizing the purpose of the application framework layer accessing the device hardware. For example, the hardware abstraction layer may be MotionHAL, which communicates with the smart sensor hub (also known as SensorHub).

[0119] The kernel layer is a layer between hardware and software. The kernel layer provided in the embodiment of the present application includes a display driver, an audio driver, and a sensor driver.

[0120] The SensorHub can connect to the audio hardware codec, obtain the audio stream, and obtain the total amount of the user's voice from the preset initial moment to the moment of obtaining the audio stream, and report the total amount of voice to the AP for display. In other examples, the SensorHub can connect to the accelerometer sensor 180E, obtain the user's motion information, and calculate the user's total number of steps, and report the total number of steps to the AP for display.

[0121] In the embodiment of the present application, the SensorHub integrates an Advanced Digital Signal Processor (ADSP). In one example, see Fig.11A , which is a software block diagram of an ADSP implementation. ADSP only includes an audio protection domain. The audio protection domain includes an audio framework, an audio virtual sensor, and a call counter. The audio framework is connected to the audio hardware encoder and communicates with the audio virtual sensor and the call counter. Therefore, the processing of voice statistics is deployed in the Sensorhub, so that the voice statistics function runs in a low power mode.

[0122] In another example, see Fig. 11B , which is another implementation framework diagram of ADSP. ADSP includes an audio protection domain and a sensor protection domain. The audio protection domain includes an audio framework for connecting an audio hardware codec to convert electrical signals into raw audio data streams and send the raw audio data streams to the sensor protection domain.

[0123] The sensor protection domain includes audio virtual sensors, call counters, and pedometers. Fig. 11B The implementation framework diagram provided integrates multiple applications such as phone counters and pedometers into the same sensor protection domain, which facilitates integrated processing. That is, it helps to realize multiple functions in one sensor protection domain chip and improve user experience.

[0124] See also Fig. 12A , this figure is a complete implementation structure block diagram of a specific voice statistics function provided in an embodiment of the present application.

[0125] The application processor includes a speech volume display application, a device management framework, Motion AIDL, and Motion HAL. The speech volume display application communicates with the SensorHub through the device management framework, Motion AIDL, and Motion HAL in turn.

[0126] Sensor Hub includes an audio protection domain, which includes an audio framework, an audio virtual sensor, and a talkie. The audio framework interacts with the talkie through the audio virtual sensor, and the talkie interacts with the Motion HAL.

[0127] The audio framework is used to obtain audio data streams from audio hardware codecs.

[0128] See also Fig. 12B , this figure is a complete implementation structure block diagram of another specific voice statistics function provided in an embodiment of the present application.

[0129] The application processor includes a speech volume display application, a device management framework, Motion AIDL, and Motion HAL. The speech volume display application communicates with the SensorHub through the device management framework, Motion AIDL, and Motion HAL in turn.

[0130] Sensor Hub includes audio protection domain and sensor protection domain. The audio protection domain includes audio framework. The sensor protection domain includes audio virtual sensor and talkie. The audio framework exchanges data with talkie through audio virtual sensor, and talkie interacts with Motion HAL.

[0131] The audio framework is used to obtain audio data stream from the audio hardware codec 145.

[0132] In addition, the embodiment of the present application also provides a functional structure diagram of a call counter.

[0133] See also Fig. 12C , which is a functional structure diagram of a call counter. The call counter includes:

[0134] The audio data acquisition unit 1201 is used to acquire the audio data stream from the audio framework through the audio data virtual sensor.

[0135] The audio data processing unit 1202 is used to pre-process the acquired audio data stream, including decompression, filtering, denoising and other processing, to obtain a pre-processed audio data stream.

[0136] The speech recognition unit 1203 is used to obtain the speech volume of the speech input from the pre-processed audio data stream. The speech recognition unit can perform recognition based on the speech recognition algorithm provided by the speech recognition algorithm unit 1204.

[0137] The speech recognition algorithm unit 1204 is used to perform recognition using the provided speech recognition algorithm.

[0138] The speech volume statistics unit 1205 is used to update the speech statistics based on the speech volume of the speech input to obtain the speech statistics at the current moment.

[0139] The speech volume reporting unit 1206 is used to obtain the speech statistics at the current moment from the speech volume statistics unit 1205 and report it to the application processor for display. Fig.13A , the figure is based on Figure 12A to Figure 12B Interaction diagram of any implementation method for voice statistics function.

[0140] S121: Turn on the speaking volume display function.

[0141] In the embodiment of the present application, opening the speaking volume display function specifically includes opening a display application and setting a display rule and a display format. For example, the display rule is one of a word-based display rule, a character-based display rule, and a sentence-based display rule. Those skilled in the art can adjust it as needed.

[0142] The display mode includes whether the display interface is in a bright screen mode or a dark screen mode. The display mode also includes only displaying the voice statistics on the lock screen interface, or displaying the voice statistics and change trends for N consecutive days on other interfaces.

[0143] S122: The speech volume display application sends an opening instruction to the device management framework.

[0144] The open command is used to activate the talk counter, audio virtual sensor and audio framework, so that the talk counter can be in the audio data stream, obtain voice statistics, and the audio framework starts to collect audio data stream. The audio virtual sensor is used to send the audio framework data to the talk counter for processing.

[0145] In the embodiment of the present application, the opening instruction includes a display rule and a display form.

[0146] S123: The device management framework sends an opening instruction to the phone through Motion AIDL and Motion HAL.

[0147] S124: The call counter sends an opening instruction to the audio virtual sensor.

[0148] S125: The audio virtual sensor sends an opening instruction to the audio framework.

[0149] S126: The audio framework collects the audio data stream.

[0150] S127: The audio framework sends the audio data stream to the audio virtual sensor.

[0151] S128: The audio virtual sensor sends the audio data stream to the call counter.

[0152] S129: The call counter processes the audio data stream and obtains voice statistics.

[0153] S1210: The call counter reports the voice statistics to the Motion HAL, and the Motion HAL uploads the voice statistics to the device management framework through the Motion AIDL.

[0154] In the embodiment of the present application, the call counter reports different voice statistics and voice statistics related information to the Motion HAL according to the opening instruction, such as voice statistics for N consecutive days, the average value of voice statistics for N consecutive days, etc.

[0155] In addition, the call counter can determine the upload frequency according to whether the display mode is in the off-screen mode or the on-screen mode. If the display mode received by the call counter is in the on-screen mode, the displayed value is reported in real time. If the display mode received by the call counter is in the off-screen mode, the call counter obtains the last reporting time, and uses the last reporting time plus the interval preset time to obtain the current reporting time of the second value reported this time.

[0156] S1211: The device management framework reports the voice statistics to the speech volume display application.

[0157] S1212: Displaying speech statistics in the speech display area.

[0158] See also Fig. 13B , which is an interactive diagram of a method for turning off the voice statistics function provided by an embodiment of the present application. The method specifically includes:

[0159] S131: The user closes the speech volume display application.

[0160] S132: The speech volume display application sends a closing instruction to the device management framework.

[0161] S133: The device management framework sends a close instruction to the Motion AIDL.

[0162] S134: Motion AIDL sends a close command to Motion HAL.

[0163] S135: Motion HAL sends a shutdown command to the phone counter.

[0164] S136: The call counter sends a close instruction to the audio virtual sensor.

[0165] S137: The audio virtual sensor sends a close instruction to the audio framework.

[0166] S138: The audio framework stops collecting audio data streams.

[0167] In addition, when the speech volume display function is continuously turned on, the display rules can be adjusted. The user opens the settings interface and adjusts the display rules, for example, from a word-based display rule to a sentence-based display rule. At this time, a conversion instruction is sent to the device management framework, and the device management framework sends the conversion instruction to the talk counter through Motion AIDL and Motion HAL. The talk counter adjusts the reported speech statistics according to the conversion instruction and displays the adjusted speech statistics in the display area of ​​the display interface.

[0168] Therefore, the embodiment of the present application deploys the processing of voice statistics in Sensorhub, so that the voice statistics function runs in low power consumption mode. The embodiment of the present application also provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, each function or step performed by the terminal 100 in the above method embodiment is implemented.

[0169] Another embodiment of the present application further provides a computer program product including instructions. When the computer program product is run on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0170] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for implementing a speech statistics function, characterized in that: Applied to a terminal, the terminal includes an application processor and an intelligent sensor hub SensorHub, the method includes: In response to the voice statistics function being in an on state and the step counting function being in an on state, the application processor displays the voice statistics in a voice display area of ​​the display interface and displays the exercise step statistics in a step display area of ​​the display interface; the displayed voice statistics are to display how many sentences, words, or characters are spoken; In response to the voice statistics function being continuously turned on, after receiving the user's voice input, the Sensorhub obtains the voice volume of the voice input; based on the voice volume of the voice input, obtains the updated voice statistics; and reports the updated voice statistics to the application processor; if the pedometer function is continuously turned on, after receiving the user's exercise information, the Sensorhub obtains the total number of exercise steps of the exercise information; and reports the total number of exercise steps to the application processor; The application processor displays the updated voice statistics in the voice display area of ​​the display interface, and displays the total number of exercise steps in the step display area.

2. The method according to claim 1, characterized in that: The terminal further includes an audio codec, and after receiving a voice input from a user, the method includes: The audio codec converts the user's voice input into an audio data stream; The Sensorhub obtains the audio data stream from the audio codec, and based on the audio data stream, obtains the voice volume of the voice input; based on the voice volume of the voice input, obtains the updated voice statistics; and reports the updated voice statistics to the application processor to display the updated voice statistics in the voice display area.

3. The method according to claim 2, characterized in that: When the audio data stream is a pulse code modulation PCM code stream, the Sensorhub obtains the audio data stream from the audio codec, and obtains the voice volume of the voice input based on the audio data stream, including: The Sensorhub obtains the PCM code stream from the audio codec; performs noise reduction processing on the PCM code stream; and obtains the voice volume of the voice input using the PCM code stream after the noise reduction processing.

4. The method according to claim 2 or 3, characterized in that: The SensorHub reports to the application processor once at a first time interval in the screen-on mode, and reports to the application processor once at a second time interval in the screen-off mode; the second time interval is greater than the first time interval.

5. The method according to claim 1, characterized in that: After receiving voice inputs from multiple users, the method further includes: Matching the target user from the voice inputs of the multiple users based on the acquired identification information of the target user; The step of displaying the updated speech statistics in the speech display area of ​​the display interface includes: The updated speech statistics of the target user are displayed in the speech display area.

6. The method according to claim 1, characterized in that: The method further comprises: obtaining speech statistics of N consecutive cycles, wherein N is an integer greater than 1; The speech statistics and change trends of the N consecutive cycles are displayed in the speech display area of ​​the display interface.

7. The method according to claim 1, characterized in that: Before the application processor responds that the voice statistics function is in an enabled state, the method includes: The application processor displays a setting interface; the setting interface includes a first function selection item of the voice statistics function, and the first function selection item is used to turn on or off the voice statistics function of the terminal; In response to an opening operation on the first function selection item of the setting interface, the voice statistics function is opened.

8. The method according to claim 7, characterized in that: The setting interface also includes a second function selection item of the voice statistics function, and the second function selection item is used to determine the display rule of the voice statistics function. The method further includes: in response to a triggering operation on the second function selection item of the setting interface, determining a display rule of the voice statistics function.

9. The method according to claim 8, characterized in that: The display rule includes one of a sentence-based display rule, a word-based display rule, and a phrase-based display rule.

10. A terminal, characterized in that: including a processor and a memory; The memory is used to store computer-executable instructions; The processor is configured to execute the computer-executable instructions stored in the memory, so that the processor executes the method according to any one of claims 1 to 9.

11. A terminal-readable storage medium, characterized in that: The terminal-readable storage medium stores a terminal program or instruction, and when the terminal program or instruction is executed, the method according to any one of claims 1 to 9 is implemented.

Images