Display method, electronic device, medium, and program product

Abstract

This application relates to the field of electronic device technology, and discloses a display method, electronic device, medium, and program product. In this method, a user's first operation is detected, which activates the electronic device's audio pickup function; a target display element is displayed, which identifies at least one of the following information about the audio data acquired by the electronic device: sound source direction, sound intensity, and pickup range. By detecting the user's first operation to activate the audio pickup function and displaying the target display element, visual feedback of the spatial characteristics of the audio data is achieved. Thus, the user can perceive information such as the sound source direction, sound intensity, and pickup range of the audio data acquired by the electronic device through the target display element, allowing for appropriate adjustments based on the audio data information and improving the user experience. For example, if the sound intensity is weak, the user can move the electronic device towards the sound source direction to increase the sound intensity.

Description

Technical Field

[0001] This application relates to the field of electronic device technology, and in particular to a display method, electronic device, medium, and program product. Background Technology

[0002] As users' needs for perceiving complex acoustic environments (such as complex scenarios with multiple sound sources from different directions) increase, multi-microphone arrays are typically deployed in mobile phones and other terminal devices.

[0003] Multi-microphone arrays can calculate spatial information such as the time delay and phase difference of sound waves arriving at different microphones, and estimate sound source information such as the direction of arrival (DOA), distance, and sound intensity through geometric operations (such as generalized cross-correlation).

[0004] However, in current electronic devices that use sound pickup functions (such as video recorders and cameras), users are usually unable to perceive the sound source information (such as sound source direction and sound intensity) corresponding to the recorded audio. This makes it impossible for users to make corresponding adjustments or other decisions based on sound source information (such as sound source direction and sound intensity) in complex acoustic environments (such as complex scenes with multiple sound sources in different directions), thus affecting the user experience. Summary of the Invention

[0005] To improve the user experience, this application provides a display method, electronic device, medium, and program product.

[0006] A first aspect of this application provides a display method applied to an electronic device. The method includes: detecting a first operation by a user, the first operation being used to activate the audio pickup function of the electronic device; and displaying a target display element, the target display element being used to identify at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and pickup range.

[0007] In this embodiment, by detecting the user's first operation to activate the sound pickup function and displaying the target display element, a visual feedback of the spatial characteristics of the audio data is achieved. Thus, the user can perceive information such as the sound source direction, sound intensity, and pickup range of the audio data collected by the electronic device through the target display element, allowing for appropriate adjustments based on the audio data information and improving the user experience. For example, if the sound intensity is weak, the user can move the electronic device towards the sound source to increase the sound intensity.

[0008] In one possible implementation of the first aspect described above, the target display element includes at least one of the following: a dial for identifying the sound pickup range and / or the sound intensity in each direction; and a pointer for indicating the dominant sound source direction of the audio data acquired by the electronic device, wherein the dominant sound source direction is the direction corresponding to the energy value of the microphone array of the electronic device in the sound field at the current moment that is greater than a preset threshold.

[0009] In this embodiment, the dial can be used to identify the sound reception range and directional intensity distribution, and the pointer can be used to indicate the direction of the dominant sound source. The two work together to achieve a structured presentation of multi-dimensional audio information.

[0010] In one possible implementation of the first aspect described above, the dial is a circular dial, and a point on the pointer is set at the center of the circular dial.

[0011] In this embodiment, the dial is set as a ring and the center of the pointer is set at the center of the dial, which conforms to the natural cognitive habit of human eyes in perceiving direction (similar to a compass or clock face). Users can understand the correspondence between the pointer's direction and the direction of the sound source without learning, which can reduce the user's cognitive load.

[0012] In one possible implementation of the first aspect above, the dial includes multiple angle segments, a first visual style parameter is greater than a second visual style parameter, the first visual style parameter is the visual style parameter of at least one angle segment among the multiple angle segments, and the second visual style parameter is the visual style parameter of at least one angle segment among the multiple angle segments other than the aforementioned at least one angle segment; wherein, the visual style parameter includes at least one of the following: color, brightness, width, and length.

[0013] In this embodiment, by dividing the dial into multiple angular segments and assigning different visual style parameters (color, brightness, width, length, etc.) to each segment, a visual mapping between spatial direction and sound intensity can be achieved. Thus, by observing the visual differences between the angular segments, users can quickly identify areas with higher sound intensity, improving the intuitiveness and efficiency of information acquisition.

[0014] In one possible implementation of the first aspect above, the color of the multiple angular segments is positively correlated with the sound intensity in the corresponding direction of the multiple angular segments; and / or, the brightness of the multiple angular segments is positively correlated with the sound intensity in the corresponding direction of the multiple angular segments; and / or, the width of the multiple angular segments is positively correlated with the sound intensity in the corresponding direction of the multiple angular segments; and / or, the length of the multiple angular segments is positively correlated with the sound intensity in the corresponding direction of the multiple angular segments.

[0015] In this embodiment, by establishing a positive correlation between visual style parameters (color, brightness, width, length, etc.) and sound intensity, a continuous visual mapping of sound intensity can be achieved. In this way, users can intuitively perceive changes in sound energy intensity in different directions. For example, the stronger the sound, the darker the color, the higher the brightness, the wider the width, or the longer the length of the corresponding segment.

[0016] In one possible implementation of the first aspect described above, at least one shape parameter of the pointer is positively correlated with the sound intensity in the direction of the dominant sound source, wherein the shape parameter includes at least one of the following: length, thickness, color, and brightness.

[0017] In this embodiment, by associating the shape parameters (length, thickness, color, brightness, etc.) of the pointer with the sound intensity of the dominant sound source, the direction of the pointer indicates the location of the sound source, and the shape of the pointer indicates the sound source intensity. In this way, the user can simultaneously obtain information on both the direction of the sound source and the sound intensity without repeatedly switching their gaze between the pointer and the level meter, thereby improving information density and cognitive efficiency.

[0018] In one possible implementation of the first aspect above, the method further includes: the sound intensity corresponding to the direction of the dominant sound source is greater than an intensity threshold, the pointer display state is a target state, and / or, triggering the electronic device to vibrate; wherein the target state includes at least one of the following: shaking, flashing, color changing.

[0019] In this embodiment, when the intensity of the dominant sound source exceeds a threshold, a multimodal user alert is achieved by changing the pointer display state (jittering, flashing, color changing) or triggering the vibration of the electronic device. For example, in a recording scenario, the user can be alerted that the current volume is too loud and may cause clipping distortion or other risks. Or, for example, in a hearing-impaired assistance scenario, the user can be alerted to the presence of high-intensity sounds in the surroundings (such as alarms, vehicle horns, etc.). Or, for example, in a meeting scenario, the user can be alerted that an important speaker has begun to speak.

[0020] In one possible implementation of the first aspect described above, the first operation is to launch a recorder application; the target display element includes a pointer; and the method further includes: detecting an interaction operation performed by the user on the pointer at a first moment, wherein the dominant sound source direction indicated by the pointer at the first moment is a first direction, wherein the interaction operation includes at least one of the following: clicking, long pressing, double-clicking; and in response to the interaction operation, performing signal enhancement processing on the audio data acquired in the first direction.

[0021] In this embodiment of the application, in the recording scenario, by detecting the user's interactive operation on the pointer (click, long press, double click), the audio in the direction pointed to by the pointer is enhanced accordingly. The user does not need to enter a complicated settings menu. He / she can simply click the pointer on the screen that indicates the dominant sound source to enhance the sound source in a specific direction, thereby improving the convenience and response speed of the operation during the recording process.

[0022] In one possible implementation of the first aspect described above, the method further includes: keeping the pointer pointing in the first direction.

[0023] In this embodiment of the application, after the user performs an interactive operation, the pointer remains pointing to the direction of enhancement, so that the user can intuitively confirm which direction of audio is being enhanced, thus avoiding repeated or erroneous operations caused by unclear interface status.

[0024] In one possible implementation of the first aspect above, the first operation is to activate the hearing-impaired assistive function; the target display element includes a dial, the dial includes multiple angle segments; and the method further includes: detecting that the sound intensity in the second direction is greater than a danger intensity threshold, flashing at least one angle segment in the dial corresponding to the second direction, and / or triggering vibration of the electronic device.

[0025] In this embodiment of the application, in a hearing-impaired assistive application scenario, when the sound intensity in a second direction exceeds a danger threshold, directional warnings of dangerous sound sources can be achieved by flashing the dial in segments at the corresponding angles or triggering device vibration. For example, hearing-impaired users may not be able to perceive sudden high-intensity sounds (such as vehicle horns or alarms) through hearing. However, visual and tactile feedback can promptly alert users to dangerous directions, thereby improving the safety and practical value of electronic devices.

[0026] In one possible implementation of the first aspect described above, the method further includes: detecting a human voice, and displaying an identity label corresponding to the human voice and / or a text summary corresponding to the human voice in a preset display area around the angular segment corresponding to the direction of the human voice on a dial.

[0027] In this embodiment, semantic visualization of the voice source's identity can be achieved by detecting human voice and displaying an identity label or text summary around the corresponding direction of the dial. For example, in a meeting scenario, the speaker's name can be displayed to facilitate meeting notes. Or, for example, in a hearing-impaired assistance scenario, the dialogue text can be displayed in real time to facilitate the user's access to the audio content. Or, for example, in an interview scenario, the interviewee's identity can be labeled to facilitate post-editing.

[0028] In one possible implementation of the first aspect described above, the method further includes: a switch control for displaying the target function, the display state of which is used to indicate whether the target function is in an activated state, wherein the target function is a function that visualizes at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and sound reception range.

[0029] In this embodiment, by displaying a switch control and indicating the activation status of the target function, users can independently enable or disable the visualization function of sound source direction, sound intensity, and reception range as needed. This avoids power consumption waste and interface redundancy caused by continuously displaying the visualization interface in unnecessary scenarios (such as simple voice calls), and users can choose whether to enable the function based on personal preference and actual usage scenarios.

[0030] A second aspect of this application provides an electronic device, comprising: a memory for storing instructions executable by one or more processing units of the electronic device; and a processor, one of the processors of the electronic device, for executing the instructions stored in the memory to implement any of the methods described in the first aspect above.

[0031] In the embodiments of this application, when the electronic device in the second aspect implements any of the methods in the first aspect, it can achieve the same or similar technical effects as any of the methods in the first aspect.

[0032] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed, implement any of the methods described in the first aspect above.

[0033] In the embodiments of this application, the computer storage medium in the third aspect can achieve the same or similar technical effects as any of the methods in the first aspect when implementing any of the methods described in the first aspect.

[0034] The fourth aspect of this application provides a computer program product including instructions that, when executed, implement any of the methods described in the first aspect above.

[0035] In the embodiments of this application, the computer program product in the fourth aspect can achieve the same or similar technical effects as any of the methods in the first aspect when implementing any of the methods described above. Attached Figure Description

[0036] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 A schematic diagram of a user interface for a recorder application is shown according to an embodiment of this application;

[0038] Figure 2A and Figure 2B A schematic diagram illustrating the changes in the user interface of a recorder application is shown according to an embodiment of this application;

[0039] Figure 3 A flowchart of a display method is shown according to an embodiment of this application;

[0040] Figure 4A and Figure 4B A schematic diagram of a dial is shown according to an embodiment of this application;

[0041] Figure 5A and Figure 5B A schematic diagram of another dial is shown according to an embodiment of this application;

[0042] Figure 6A and Figure 6B A schematic diagram of a dial shape is shown according to an embodiment of this application;

[0043] Figure 7 A schematic diagram of a user interface for a recorder application that displays human voice information is shown according to an embodiment of this application;

[0044] Figure 8 A schematic diagram of the structure of an electronic device 100 is shown according to an embodiment of this application;

[0045] Figure 9 A schematic diagram of the structure of a display device is shown according to an embodiment of this application. Detailed Implementation

[0046] The illustrative embodiments of this application include, but are not limited to, a display method, an electronic device, a medium, and a program product.

[0047] As mentioned earlier, in current electronic devices that use sound pickup functions (such as video recorders, cameras, etc.), users are usually unable to perceive the sound source information (such as sound source direction, sound intensity, etc.) corresponding to the recorded audio. This makes it impossible for users to make corresponding adjustments or other decisions based on sound source information (such as sound source direction, sound intensity, etc.) in complex acoustic environments (such as complex scenarios with multiple sound sources in different directions), thus affecting the user experience.

[0048] For example, taking a recorder application as an example, Figure 1 A schematic diagram of the user interface for a recorder application is shown.

[0049] like Figure 1 As shown, the user interface 10 includes an audio display area 101 and a recording status display area 102. The audio display area 101 displays information about one or more recorded audio data. The recording status display area 102 includes display elements such as a "speech-to-text" control 1021, a "voice enhancement" control 1022, an audio track display area 1023, a recording control group 1024, a digital timer, and storage status prompts. The recording control group 1024 includes controls for pausing / continuing audio recording, ending audio recording, and adding tags.

[0050] It is understandable that users cannot perceive information such as the direction of the sound source and the sound intensity of the recorded audio data through the user interface 10.

[0051] In view of this, embodiments of this application provide a display method. The method includes: detecting a first operation by a user, the first operation being used to activate the sound pickup function of an electronic device; displaying a target display element, the target display element being used to identify at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and sound pickup range.

[0052] It's understandable that by detecting the user's first action to activate the sound pickup function and displaying the target element, a visual feedback of the spatial characteristics of the audio data is achieved. In this way, users can perceive information such as the direction of the sound source, sound intensity, and pickup range of the audio data collected by the electronic device through the target display element, allowing them to make corresponding adjustments based on the audio data information and improve the user experience. For example, if the sound intensity is weak, the user can move the electronic device towards the sound source to increase the sound intensity.

[0053] For example, taking the recording scenario as an example, refer to Figure 2A and Figure 2B The diagram shows the changes to the user interface of the recorder application.

[0054] like Figure 2AAs shown, the user interface 20a includes an audio display area 201 and a recording status display area 202. The recording status display area 202 includes display elements such as a "speech-to-text" control 2021, a "voice enhancement" control 2022, an audio track display area 2023, a recording control group 2024, a "pickup direction" function switch control 2025, a digital timer, and storage status prompts.

[0055] Understandable, compared to Figure 1 The user interface 10 and user interface 20a shown here have added a "pickup pointing" function switch control 2025, and the "pickup pointing" function switch control 2025 is in an inactive state. Users can enable the function corresponding to the "pickup pointing" function switch control 2025 through interactive operations (such as clicking, long pressing, etc.).

[0056] For example, if a user's interactive click operation on the "pickup direction" function switch control 2025 in the user interface 20a is detected, the electronic device 100 displays as follows: Figure 2B The user interface 20b is shown.

[0057] like Figure 2B As shown, the user interface 20b includes an audio display area 201 and a recording status display area 202. The recording status display area 202 includes display elements such as a "speech-to-text" control 2021, a "voice enhancement" control 2022, a recording control group 2024, a "pickup pointing" function switch control 2025, a pointing display element 2026, a digital timer, and storage status prompts.

[0058] The pointing display element 2026 includes a dial 20261 and a pointer 20262. The dial 20261 is used to identify the sound reception range and sound intensity in various directions, while the pointer 20262 is used to indicate the dominant sound source direction of the audio data acquired by the electronic device 100 (i.e., the direction the pointer is pointing). The dial 20261 is a circular dial, and a point on the pointer 20262 is set at the center of the circular dial. Display elements such as a digital timer and storage status prompts are displayed in a preset display area on the dial 20261.

[0059] Furthermore, the dial 20261 includes multiple angle segments, and the color of the angle segment 20263 corresponding to the preset range of the direction indicated by the pointer 20262 is darker than the color of other angle segments in the dial 20261 except for the angle segment 20263 corresponding to the preset range of the direction indicated by the pointer.

[0060] Understandable, compared to Figure 2A User interface 20a shown Figure 2BIn the user interface 20b, the recording status display area 202 displays the pointing display element 2026 instead of the audio track display area 2023, and the "pickup pointing" function switch control 2025 in the user interface 20b is in the on state (for example, the color of the "pickup pointing" function switch control 2025 in the user interface 20b is darker than the color of the "pickup pointing" function switch control 2025 in the user interface 20a).

[0061] It is understandable that users can perceive information such as the direction of the sound source and the sound intensity of the recorded audio data through the user interface 20b.

[0062] Understandable. Figure 2A and Figure 2B The user interface changes shown are only described using the recording scenario of a recorder application as an example to illustrate the form of the target display element (pointing to display element 2026). The display method provided in this application embodiment is not limited to the recording scenario of a recorder application. The display method provided in this application embodiment can also be applied to other scenarios that use the sound pickup function of electronic devices, such as hearing-impaired assistance scenarios, conference scenarios, etc. This application does not impose any restrictions on this.

[0063] To better understand the technical solutions of the embodiments of this application, some technical solutions of this application will be described in detail below.

[0064] Figure 3 A flowchart illustrating a display method is shown according to an embodiment of this application. It can be understood that... Figure 3 The process shown is executed by electronic device 100. For simplicity, the following description... Figure 3 The execution entity will not be described again when the process is shown.

[0065] like Figure 3 As shown, this process includes, but is not limited to:

[0066] S301: The user's first operation was detected. The first operation was used to activate the electronic device's sound pickup function.

[0067] In some embodiments, the first operation includes at least one of the following: launching a recorder application, launching a hearing-impaired assistive function, or launching a recording function of a conference application.

[0068] S302: Display target display element, the target display element is used to identify at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and sound reception range.

[0069] In some embodiments, the target display element includes at least one of the following: a dial for identifying the sound pickup range and / or the sound intensity in each direction; and a pointer for indicating the dominant sound source direction of the audio data acquired by the electronic device, wherein the dominant sound source direction is the direction corresponding to the energy value of the microphone array of the electronic device in the sound field at the current moment that is greater than a preset threshold.

[0070] It is understandable that by defining the dominant sound source direction as the direction of the sound source with an energy value greater than a preset threshold, the frequent erroneous movement of the pointer due to environmental noise or weak interference can be avoided, thereby improving the stability and readability of the display.

[0071] In other embodiments, the dominant sound source direction is the direction corresponding to the maximum energy value in the sound field of the microphone array of the electronic device at the current moment, and this application does not limit this.

[0072] For example, refer to the above Figure 2B The pointing display element 2026 shown is an example of a target display element. The pointing display element 2026 includes a dial 20261 and a pointer 20262.

[0073] It is understandable that the dial is used to identify the sound reception range and directional intensity distribution, while the pointer is used to indicate the direction of the dominant sound source. The two work together to achieve a structured presentation of multi-dimensional audio information.

[0074] In some embodiments, the dial is a circular dial, and a point on the pointer is located at the center of the circular dial. For example, see reference... Figure 2B The pointers shown are the dial 20261 and pointer 20262 in the display element 2026.

[0075] It's understandable that setting the dial to a ring shape and placing the pointer's center at the center of the dial aligns with the human eye's natural cognitive habits regarding direction (similar to a compass or clock face). Users can understand the correspondence between the pointer's direction and the sound source without needing to learn, thus reducing their cognitive load.

[0076] In some embodiments, the dial includes multiple angle segments, a first visual style parameter is greater than a second visual style parameter, the first visual style parameter is the visual style parameter of at least one angle segment among the multiple angle segments, and the second visual style parameter is the visual style parameter of at least one angle segment among the multiple angle segments excluding the at least one angle segment; wherein, the visual style parameter includes at least one of the following: color, brightness, width, and length.

[0077] For example, refer to Figure 2BThe color of the angle segment 20263 corresponding to the preset range of the direction indicated by the pointer 20262 in the dial 20261 of the user interface 20b shown (as an example of a first visual style) is darker than the colors of other angle segments in the dial 20261 other than the angle segment 20263 corresponding to the preset range of the direction indicated by the pointer (as an example of a second visual style).

[0078] For example, refer to Figure 4A and Figure 4B A schematic diagram of the dial shown.

[0079] like Figure 4A As shown, the length of the angle segment 401 corresponding to the preset range of the direction indicated by the pointer (as an example of the first visual style) is longer than the length of other angle segments on the dial besides the angle segment 401 corresponding to the preset range of the direction indicated by the pointer (as an example of the second visual style).

[0080] like Figure 4B As shown, the width of the angle segment 402 corresponding to the preset range of the direction indicated by the pointer (as an example of the first visual style) is wider than the width of other angle segments on the dial besides the angle segment 402 corresponding to the preset range of the direction indicated by the pointer (as an example of the second visual style).

[0081] It's understandable that by dividing the dial into multiple angular segments, and presenting different visual style parameters (color, brightness, width, length, etc.) for each segment, a visual mapping between spatial direction and sound intensity can be achieved. In this way, users can quickly identify areas with higher sound intensity by observing the visual differences between the various angular segments, improving the intuitiveness and efficiency of information acquisition.

[0082] In some embodiments, the color of the multiple angular segments is positively correlated with the sound intensity in the direction corresponding to the multiple angular segments; and / or, the brightness of the multiple angular segments is positively correlated with the sound intensity in the direction corresponding to the multiple angular segments; and / or, the width of the multiple angular segments is positively correlated with the sound intensity in the direction corresponding to the multiple angular segments; and / or, the length of the multiple angular segments is positively correlated with the sound intensity in the direction corresponding to the multiple angular segments.

[0083] For example, refer to Figure 5A and Figure 5B A schematic diagram of the dial shown.

[0084] like Figure 5A As shown, the sound intensity is greatest in the direction indicated by the pointer, and the closer the angle segment is to the direction indicated by the pointer, the longer the length of the angle segment.

[0085] like Figure 5BAs shown, the sound intensity is greatest in the direction indicated by the pointer, and the closer the angle segment is to the direction indicated by the pointer, the wider the angle segment is.

[0086] It is understandable that by establishing a positive correlation between visual style parameters (color, brightness, width, length, etc.) and sound intensity, a continuous visual mapping of sound intensity can be achieved. In this way, users can intuitively perceive changes in sound energy intensity from different directions.

[0087] For example, the greater the sound intensity in the direction the pointer points to (the direction of the dominant sound source), the longer the pointer. Or, for example, the greater the sound intensity in the direction the pointer points to (the direction of the dominant sound source), the wider the pointer. Or, for example, the greater the sound intensity in the direction the pointer points to (the direction of the dominant sound source), the darker the pointer color. Or, for example, the greater the sound intensity in the direction the pointer points to (the direction of the dominant sound source), the brighter the pointer.

[0088] In some embodiments, at least one shape parameter of the pointer is positively correlated with the sound intensity in the direction of the dominant sound source, wherein the shape parameter includes at least one of the following: length, thickness, color, and brightness.

[0089] It can be understood that by associating the shape parameters of the pointer (length, thickness, color, brightness, etc.) with the sound intensity of the dominant sound source, the direction of the pointer indicates the location of the sound source, and the shape of the pointer indicates the sound source intensity. In this way, users can obtain information on both the direction of the sound source and the sound intensity simultaneously without having to repeatedly switch their gaze between the pointer and the level meter, thereby improving information density and cognitive efficiency.

[0090] It is understood that the description of the dial shape (e.g., a circular dial) in the above embodiments is merely illustrative and does not constitute the only limitation on the dial shape. The specific shape of the dial can be flexibly designed and adjusted according to the actual application scenario, device form, or user interaction habits. For example, the dial can also be presented as a semi-circle, fan shape, arc shape, rectangle, polygonal outline, irregular free curve shape, etc. As long as the shape of the dial can clearly divide different angular segments and can represent the sound intensity in the corresponding direction through changes in visual style parameters (such as color, brightness, width, etc.), or can cooperate with pointers and other indicator elements to indicate the direction of the sound source, it falls within the technical solutions covered by this application.

[0091] For example, refer to Figure 6A and Figure 6B A schematic diagram showing the shape of the dial.

[0092] like Figure 6A As shown, the dial is rectangular. Figure 6B As shown, the dial is a regular hexagon.

[0093] In this way, the display effect of the target display element can be better adapted to different types of electronic devices (such as foldable screen phones, smartwatches, in-vehicle terminal screens, etc.) and the diverse display areas in electronic devices, thereby improving the user experience and having wider applicability.

[0094] Understandable. Figure 3 The implementation process shown is only an illustrative example; in other embodiments, Figure 3 The process shown may include more or fewer steps, and this application does not limit this.

[0095] For example, in some embodiments, in the above Figure 3 In the method shown, when the sound intensity corresponding to the direction of the dominant sound source is greater than the intensity threshold, the pointer display state is the target state, and / or, the electronic device 100 is triggered to vibrate; wherein, the target state includes at least one of the following: shaking, flashing, color changing.

[0096] It's understandable that when the intensity of the dominant sound source exceeds a threshold, multimodal user alerts can be achieved by changing the pointer display state (e.g., shaking, flashing, color changing) or triggering the electronic device to vibrate 100%. For example, in a recording scenario, it can alert the user that the current volume is too loud and may cause clipping distortion. Or, for example, in a hearing-impaired scenario, it can alert the user to the presence of high-intensity sounds in the surroundings (e.g., alarms, vehicle horns). Or, for example, in a meeting scenario, it can alert the user that an important speaker is about to begin speaking.

[0097] For example, in some embodiments, corresponding to the first operation being the operation of launching a recorder application, i.e., applied to a recording scenario, if the electronic device 100 detects an interactive operation performed by the user on the pointer at the first moment; in response to the interactive operation, signal enhancement processing is performed on the audio data collected in the first direction. Wherein, the dominant sound source direction indicated by the pointer at the first moment is the first direction, and the interactive operation includes at least one of the following: click, long press, double click.

[0098] In other words, during recording, by detecting user interactions with the pointer (such as tapping, long pressing, double-tapping, etc.), the audio signal in the direction the pointer points is correspondingly amplified. Thus, users don't need to navigate through complex settings menus; they can simply tap the pointer on the screen indicating the direction of the dominant sound source to amplify the sound from that direction, improving the ease of operation and responsiveness during recording.

[0099] In other embodiments, the pointer may remain pointing in the first direction after the user performs an interactive operation.

[0100] In other words, after the user performs an interactive operation, the pointer remains pointing in the direction of enhancement. This allows the user to intuitively confirm which direction's audio is being enhanced, avoiding duplicate or accidental operations due to unclear interface status.

[0101] Or, for example, in some embodiments, corresponding to the first operation being the activation of a hearing-impaired assistive function, i.e., applied to a hearing-impaired assistive application scenario, if the electronic device 100 detects that the sound intensity in the second direction is greater than the danger intensity threshold, at least one angle segment in the dial corresponding to the second direction flashes, and / or, triggers the vibration of the electronic device.

[0102] In other words, in hearing-impaired applications, when the electronic device 100 detects that the sound intensity in a second direction exceeds a danger threshold, it can provide directional warnings of dangerous sound sources by flashing the dial in segments at the corresponding angles or triggering device vibration. For example, hearing-impaired users may not be able to perceive sudden high-intensity sounds (such as vehicle horns or alarms) through hearing. However, visual and tactile feedback can promptly alert users to the direction of danger, thereby improving the safety and practical value of the electronic device 100.

[0103] Or, for example, in some embodiments, if the electronic device 100 detects a human voice, it displays the identity tag corresponding to the human voice and / or the text summary corresponding to the human voice in a preset display area around the angle segment corresponding to the direction of the human voice on the dial.

[0104] It's understandable that by detecting human voices and displaying identity tags or text summaries around the corresponding direction on a dial, semantic visualization of the voice source's identity can be achieved. For example, in a meeting scenario, the speaker's name can be displayed, facilitating meeting note-taking. Or, for example, in a hearing-impaired scenario, the dialogue text can be displayed in real time, making it easier for users to access the audio content. Or, for example, in an interview scenario, the interviewee's identity can be labeled, facilitating post-editing.

[0105] For example, refer to Figure 7 The diagram shows a user interface for a voice recorder application that displays human voice information.

[0106] like Figure 7 As shown, user interface 70a is the user interface for the recorder application. The display area around the angle segments above the dial 701 displays the identification label "small h" and the corresponding text summary of the voice. The display area around the angle segments on the left side of the dial 701 displays the identification label "small w" and the corresponding text summary of the voice. In this way, the user can intuitively see which direction "small h" and "small w" are speaking.

[0107] It is understood that in some embodiments, the electronic device 100 may also display a switch control for the target function (e.g., as described above). Figure 2B The “pickup direction” function switch control 2025 shown indicates whether the target function is in the activated state. The target function is to visualize at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and sound pickup range.

[0108] It's understandable that by displaying a switch control and indicating the activation status of the target function, users can independently enable or disable the visualization function for sound source direction, sound intensity, and reception range as needed. This avoids the power waste and interface redundancy caused by continuously displaying the visualization interface in unnecessary scenarios (such as simple voice calls), and users can choose whether to enable this function based on personal preference and actual usage scenarios.

[0109] It is understood that the display method provided in this application can be applied to electronic devices 100 such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), or specialized cameras (such as SLR cameras and point-and-shoot cameras). This application does not impose any limitations on the specific type of the electronic device 100.

[0110] For example, Figure 8 A schematic diagram of the structure of an electronic device 100 is shown according to an embodiment of this application.

[0111] like Figure 8 As shown, the electronic device 100 may include a processor 110, a memory 120, an interface module 130, a power module 140, a wireless communication module 150, a mobile communication module 160, an audio module 170, a sensor module 180, buttons 190, a camera 191, a display screen 192, etc.

[0112] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0113] The processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU).

[0114] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory.

[0115] In this embodiment, the processor 110 can control the fetching and execution of instructions from the memory via a controller, so that the electronic device 100 can perform the above-described operations. Figure 1 The functions performed by the electronic device 100 in the illustrated implementation process are used to realize the display method provided in this application.

[0116] The power module 140 is connected to the processor 110 and provides power to the processor 110, memory 120, camera 191, display screen 192, and mobile communication module 160.

[0117] The wireless communication function of electronic device 100 can be implemented through wireless communication module 150, mobile communication module 160, antenna, modem processor and baseband processor, etc.

[0118] Audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Audio module 170 can also be used for encoding and decoding audio signals.

[0119] The sensor module 180 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer, a distance sensor, a proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc. In some embodiments, the electronic device 100 may detect user actions such as initial operations and interactive operations through the sensor module 180 (e.g., pressure sensors, touch sensors, etc.).

[0120] Electronic device 100 implements display functions through GPU, display screen 192, and application processor.

[0121] Display screen 192 is used to display images, videos, etc. Display screen 192 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays screens 192, where N is a positive integer greater than 1. In some embodiments, electronic device 100 displays target display elements through the displays screens 192.

[0122] Electronic device 100 can perform shooting functions through ISP, camera 191, video codec, GPU, display 192 and application processor.

[0123] Camera 191 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 191, where N is a positive integer greater than 1.

[0124] The memory 120 can be used to store computer executable program code, which includes instructions. The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.). Furthermore, the memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the memory 120 and / or instructions stored in memory disposed in the processor.

[0125] Button 190 includes the power button, volume buttons, etc. Button 190 can be a mechanical button or a touch button.

[0126] It is understood that the structure illustrated in this application does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0127] It is understood that this application embodiment also provides a display device for implementing the display method provided in this application embodiment.

[0128] For example, Figure 9 A schematic diagram of the structure of a display device is shown according to an embodiment of this application.

[0129] like Figure 9 As shown, the display device 900 includes a processing module 910 and a display module 920.

[0130] The processing module 910 is used to detect the user's first operation, which is used to activate the electronic device's sound pickup function.

[0131] Display module 920 is used to display target display elements, which are used to identify at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and sound reception range.

[0132] The processing module 910 is also used to, when the sound intensity in the direction of the dominant sound source is greater than the intensity threshold, display the pointer in the target state, and / or trigger the electronic device to vibrate; wherein the target state includes at least one of the following: shaking, flashing, color changing.

[0133] The processing module 910 is also used to detect the interactive operation performed by the user on the pointer at the first moment, wherein the dominant sound source direction indicated by the pointer at the first moment is the first direction, wherein the interactive operation includes at least one of the following: click, long press, double click; and in response to the interactive operation, to perform signal enhancement processing on the audio data collected in the first direction.

[0134] The display module 920 is also used to detect human voices and display the identity label corresponding to the human voice and / or the text summary corresponding to the human voice in a preset display area around the angle segment corresponding to the direction of the human voice in the dial.

[0135] It is understood that the functions of each module in the display device 900 provided in this application embodiment can be specifically referred to the foregoing. Figure 3 The relevant descriptions in the implementation process shown are not repeated here.

[0136] It is understood that the structure illustrated in this application does not constitute a specific limitation on the display device 900. In other embodiments, the display device 900 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements.

[0137] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps performed by the electronic device in the above method embodiments, or the steps performed by the human-computer interaction module and the computing module.

[0138] This application also provides a computer program product that, when run on a terminal device, enables the terminal device to perform the steps executed by the electronic device in the above method embodiments.

[0139] This application also provides a chip, which includes a processor coupled to a memory for executing computer programs or instructions stored in the memory, thereby enabling the chip to achieve... Figure 3 The method flow is shown.

[0140] The term "user interface (UI)," used in the specification and accompanying drawings of this application, refers to the medium through which an application or operating system interacts and exchanges information with the user. It facilitates the conversion between the internal form of information and a form acceptable to the user. The user interface of an application is written in source code using specific computer languages ​​such as Java or Extensible Markup Language (XML). This source code is parsed and rendered on the terminal device, ultimately presenting user-recognizable content, such as images, text, and buttons. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, images, and text. The attributes and content of controls in the interface are defined using tags or nodes, such as XML tags. <textview> 、 <imgview> 、 <videoview>Nodes define the controls contained in the interface. A node corresponds to a control or property in the interface, and after parsing and rendering, the node is presented as the content visible to the user. In addition, many applications, such as hybrid applications, often contain web pages within their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. Web pages are source code written in a specific computer language, such as Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), JavaScript (JS), etc. Web page source code can be loaded and displayed as user-readable content by a browser or a web page display component with browser-like functionality. The specific content contained in a web page is also defined through tags or nodes in the web page source code; for example, HTML uses tags or nodes to define the content. 、 、 <video> 、 <canvas>Used to define the elements and attributes of a webpage.

[0141] The most common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an interface element such as an icon, window, or control displayed on the screen of an electronic device. The control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0143] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0144] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining..." or "in response to determining..." or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

[0145] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A display method characterized by comprising: Applied to electronic devices, the method includes: The user's first operation is detected, and the first operation is used to activate the sound pickup function of the electronic device; The target display element is used to identify at least one of the following information of the audio data collected by the electronic device: sound source direction, sound intensity, and sound reception range.

2. The method of claim 1, wherein, The target display element includes at least one of the following: A dial, the dial being used to indicate the sound reception range, and / or the sound intensity in each direction; A pointer is used to indicate the dominant sound source direction of the audio data collected by the electronic device, wherein the dominant sound source direction is the direction corresponding to the energy value of the microphone array of the electronic device in the sound field at the current moment that is greater than a preset threshold.

3. The method of claim 2, wherein, The dial is a circular dial, and a point on the pointer is located at the center of the circular dial.

4. The method according to claim 2 or 3, characterized in that, The dial includes multiple angle segments, a first visual style parameter is greater than a second visual style parameter, the first visual style parameter is the visual style parameter of at least one of the multiple angle segments, and the second visual style parameter is the visual style parameter of at least one of the multiple angle segments excluding the at least one angle segment; wherein... The visual style parameters include at least one of the following: color, brightness, width, and length.

5. The method according to claim 4, characterized in that, The colors of the multiple angular segments are positively correlated with the sound intensity in the corresponding directions of the multiple angular segments; and / or, The brightness of the plurality of angular segments is positively correlated with the sound intensity in the corresponding directions of the plurality of angular segments; and / or, The width of the plurality of angular segments is positively correlated with the sound intensity in the corresponding direction of the plurality of angular segments; and / or, The lengths of the multiple angular segments are positively correlated with the sound intensity in the corresponding directions of the multiple angular segments.

6. The method according to any one of claims 2 to 5, characterized in that, At least one shape parameter of the pointer is positively correlated with the sound intensity in the direction of the dominant sound source, wherein the shape parameter includes at least one of the following: length, thickness, color, and brightness.

7. The method according to any one of claims 2 to 6, characterized in that, The method further includes: When the sound intensity corresponding to the direction of the dominant sound source is greater than an intensity threshold, the pointer's display state is the target state, and / or, the electronic device is triggered to vibrate; wherein, The target state includes at least one of the following: shaking, flashing, and color changing.

8. The method according to any one of claims 2 to 7, characterized in that, The first operation is to launch the recorder application; the target display element includes the pointer; and, The method further includes: The system detects an interactive operation performed by the user on the pointer at a first moment, wherein the direction of the dominant sound source indicated by the pointer at the first moment is the first direction, and the interactive operation includes at least one of the following: click, long press, double click; In response to the interactive operation, the audio data acquired in the first direction is subjected to signal enhancement processing.

9. The method of claim 8, wherein, The method further includes: The pointer remains pointing in the first direction.

10. The method according to any one of claims 2 to 8, characterized in that, The first operation is to activate the hearing-impaired assistance function; the target display element includes a dial, and the dial includes multiple angle segments; and, The method further includes: If the sound intensity in the second direction is detected to be greater than the danger intensity threshold, the dial will flash at least one segment at the corresponding angle in the second direction, and / or trigger the electronic device to vibrate.

11. The method of claim 10, wherein, The method further includes: The system detects human voices and displays the identity tag corresponding to the human voice and / or the text summary corresponding to the human voice in a preset display area around the angle segment corresponding to the direction of the human voice on the dial.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: The switch control displays a target function, and the display state of the switch control is used to indicate whether the target function is in an activated state. The target function is a function that visualizes at least one of the following information from the audio data collected by the electronic device: sound source direction, sound intensity, and sound reception range.

13. An electronic device, characterized in that, include: A memory for storing instructions executed by one or more processing units of the electronic device; The processor is one of the processors of the electronic device, used to execute instructions stored in the memory to implement the method of any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, implement the method of any one of claims 1 to 12.

15. A computer program product, characterised in that, The computer program product includes instructions that, when executed, implement the method of any one of claims 1 to 12.

Images