An audio data processing method

Abstract

The application discloses an audio data processing method, comprising: providing an application program list to a user by a user terminal, wherein the application program list comprises an application program identifier and an identifier of a file type that each application program can provide; receiving selection information provided by the user by the user terminal, wherein the selection information is generated by the user based on the application program list; determining one or more application programs running in the user terminal by the user terminal; comparing the one or more application programs running with the selection information by the user terminal to mark the one or more application programs; receiving one or more files by the user terminal; selecting one or more preselected files to be used by the marked one or more application programs from the one or more files by the user terminal. The application solves the problem that the prior art cannot automatically open or automatically close a loudspeaker for specific audio files flexibly.

Description

[0001] This application is a divisional application with application number 202311085778.2, filed on August 26, 2023, and the invention titled at the time of filing was "A data processing method, apparatus and system". Technical Field

[0002] This invention relates to the field of data processing technology, and in particular to an audio data processing method. Background Technology

[0003] With the widespread adoption of 5G technology, the audio processing capabilities of smartphones are receiving increasing attention. For example, many users primarily utilize their smartphones for multimedia entertainment (such as listening to the radio, watching videos, and playing games), while others rely on audio functionality for communication purposes (e.g., using WeChat's voice messaging function, or making video or audio calls). However, under certain circumstances, smartphones may become uncontrollable when processing audio, causing audio data to play when it shouldn't, leading to a degraded user experience. Summary of the Invention

[0004] To address the problems of existing technologies, this invention provides an audio data processing method, the method comprising:

[0005] The user terminal provides the user with a list of applications, which includes application identifiers and identifiers of the file types that each application can provide;

[0006] The user terminal receives selection information provided by the user, where the selection information is generated by the user based on a list of applications;

[0007] The user terminal determines one or more applications that are running on it.

[0008] The user terminal compares one or more running applications with the selection information to identify one or more applications;

[0009] One or more files are received by the user terminal;

[0010] The user terminal selects one or more pre-selected files from one or more files to be used by one or more tagged applications.

[0011] Furthermore, data processing methods also include:

[0012] The user terminal selects one or more tagged files from one or more pre-selected files based on selection information.

[0013] The user terminal receives operation instructions from the user, wherein the operation instructions instruct the application to be run, and the operation instructions also instruct a file to be run;

[0014] The user terminal determines whether the application indicated by the operation command is one or more marked applications;

[0015] If the user terminal determines that the application indicated by the operation command is one or more marked applications, the user terminal shall continue to determine whether the file to be run indicated by the operation command is one or more marked files.

[0016] If the user terminal determines that the file to be run is one of one or more marked files, then the user terminal determines whether the user has enabled the silent mode.

[0017] Furthermore, data processing methods also include:

[0018] If the user terminal determines that the user has enabled the silent mode, then the user terminal will activate the ambient sound monitoring device set on the user terminal.

[0019] The ambient sound around the user terminal is monitored by an ambient sound monitoring device.

[0020] The ambient sound monitoring device determines whether the first ambient sound decibel level is higher than the ambient sound decibel threshold.

[0021] If the ambient sound monitoring device determines that the first ambient sound decibel is higher than the ambient sound decibel threshold, the user terminal will turn off the ambient sound monitoring device and run a file based on the operation command.

[0022] Furthermore, data processing methods also include:

[0023] If the ambient sound monitoring device determines that the first ambient sound decibel is lower than the ambient sound decibel threshold, the user terminal will continue to turn on the ambient sound monitoring device and start running a file to be run.

[0024] After the user terminal starts running a file, the ambient sound monitoring device continues to monitor the second ambient sound around the user terminal.

[0025] The ambient sound monitoring device determines whether the second ambient sound decibel level is higher than the forced silence decibel threshold.

[0026] If the ambient sound monitoring device determines that the second ambient sound decibel is higher than the forced silence decibel threshold, the mute component set in the user terminal directly sends a shutdown command to the speaker component of the user terminal. The shutdown command sent by the mute component can override the control commands of the user terminal's CPU to the speaker component.

[0027] Furthermore, data processing methods also include:

[0028] If the user terminal determines that the user has turned off the silent mode, then the user terminal will turn off the ambient sound monitoring device and the mute component set on the user terminal.

[0029] After the user terminal disables the ambient sound monitoring device and mute component set on the user terminal, the user terminal starts running the file to be executed.

[0030] Furthermore, data processing methods also include:

[0031] If the user terminal determines that the file to be run is not one of the one or more marked files, then the user terminal shall turn off the ambient sound monitoring device and the mute component set on the user terminal.

[0032] After the user terminal disables the ambient sound monitoring device and mute component set on the user terminal, the user terminal starts running the file to be executed.

[0033] Furthermore, data processing methods also include:

[0034] The user terminal determines whether the user has downloaded a new application;

[0035] After a new application is installed on the user terminal, the user terminal sends an updated application list to the user. The updated application list only includes the identifier of the new application and the identifier of the file types that the new application can provide.

[0036] The user terminal receives update selection information provided by the user, which is generated by the user based on the list of applications to be updated.

[0037] The user terminal updates the selection information using the updated application list.

[0038] The present invention also provides a data processing apparatus, which includes a unit for performing the following operations:

[0039] The user terminal provides the user with a list of applications, which includes application identifiers and identifiers of the file types that each application can provide;

[0040] The user terminal receives selection information provided by the user, where the selection information is generated by the user based on a list of applications;

[0041] The user terminal determines one or more applications that are running on it.

[0042] The user terminal compares one or more running applications with the selection information to identify one or more applications;

[0043] One or more files are received by the user terminal;

[0044] The user terminal selects one or more pre-selected files from one or more files to be used by one or more tagged applications.

[0045] Furthermore, the data processing apparatus also includes a unit for performing the following operations:

[0046] The user terminal selects one or more tagged files from one or more pre-selected files based on selection information.

[0047] The user terminal receives operation instructions from the user, wherein the operation instructions instruct the application to be run, and the operation instructions also instruct a file to be run;

[0048] The user terminal determines whether the application indicated by the operation command is one or more marked applications;

[0049] If the user terminal determines that the application indicated by the operation command is one or more marked applications, the user terminal shall continue to determine whether the file to be run indicated by the operation command is one or more marked files.

[0050] If the user terminal determines that the file to be run is one of one or more marked files, then the user terminal determines whether the user has enabled the silent mode.

[0051] The present invention also provides a data processing system, which includes a processor and a memory. The memory stores computer-executable instructions, which, when executed by the processor, cause the system to perform the data processing method described above.

[0052] Compared with existing technologies, this invention has the following advantages: When a smartphone processes audio, under certain circumstances, it may become uncontrollable, causing audio data to play when it shouldn't, resulting in a degraded user experience. Current technologies primarily address this problem by requiring the user to manually activate the smartphone's silent mode, thus shutting down the speaker. This operation causes the phone to lose all audio playback functionality. This method has significant drawbacks in the following situations: for example, the user may anticipate that certain audio playback will not (or is highly unlikely to) cause discomfort to the user and those around them, while other audio playback will. In such cases, existing solutions cannot automatically turn on the speaker for certain audio and turn it off for others. When playing audio, the user needs to adjust the speaker on / off for each audio playback event, leading to complex operations. The solution proposed in this invention aims to solve the problems existing in the prior art. Attached Figure Description

[0053] Figure 1 This is a device logic block diagram of an embodiment of the present invention.

[0054] Figure 2 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0055] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0056] As described in the background section, under certain circumstances, smartphones may become uncontrollable when processing audio, causing audio data to play when it shouldn't, resulting in a degraded user experience. For example, a smartphone might be connected to Bluetooth headphones, transmitting audio signals to the user. However, the smartphone might experience high CPU usage and insufficient memory due to running multiple applications simultaneously. This could lead to a Bluetooth connection interruption (e.g., the high CPU usage and insufficient memory could cause the phone's internal Bluetooth driver to be incorrectly interrupted, resulting in a Bluetooth connection loss). In this situation, due to the loss of Bluetooth connectivity, the phone automatically switches to speaker playback mode, playing the audio content to the external environment. If the content being played is incompatible with the environment (e.g., in a meeting setting, a speaker is speaking, and the user's phone suddenly plays audio), this will severely degrade the user experience. Furthermore, in this situation, because the CPU is at high usage, the user cannot quickly mute the speaker or audio through the phone's controls. Current technology primarily addresses this issue by requiring the user to manually activate the smartphone's silent mode, thus shutting down the speaker. This operation causes the phone to lose all audio playback functionality. This method has significant drawbacks in the following situations: for example, users can predict that the playback of certain audio will not (or is highly unlikely to) cause discomfort to themselves and those around them, while the playback of other audio will. In such cases, existing solutions cannot automatically turn on the speakers for certain audio and turn them off for others. When playing audio, users need to adjust the speaker on / off state for each audio playback event, leading to complex operations. The solution proposed in this invention aims to solve the problems existing in the prior art.

[0057] Example 1

[0058] Figure 1 This is a device logic block diagram of an embodiment of the present invention. As shown in the figure, the smartphone provided by the present invention (wherein, a smartphone is an example of a user terminal described below) includes at least a CPU, a speaker, an ambient sound monitoring device, and a mute component. Those skilled in the art should understand that... Figure 1 Only modules closely related to the technical solution of this invention are shown; other well-known modules of smartphones (e.g., power supply, memory, memory card, touch screen, transceiver, etc.) are not included. Figure 1As shown in the diagram. In this invention, the ambient sound monitoring device can be any hardware capable of measuring the decibel level of sound in the environment. The mute component can be a separate control circuit that only receives on / off instructions from the CPU. This mute component can send a shutdown command directly to the speaker without going through the CPU, thereby preventing the problem of the user being unable to turn off the speaker due to excessive CPU usage. Furthermore, the mute component is essentially only used to send the speaker shutdown command, so the cost of the mute component itself is extremely low, and adding this component does not essentially increase the cost of the smartphone. Therefore, the solution of this invention has superior market prospects.

[0059] Example 2

[0060] Figure 2 This is a flowchart of a method according to an embodiment of the present invention. As shown in the figure, the data processing method of the present invention includes the following steps:

[0061] Step 1: The user terminal provides the user with a list of applications, which includes application identifiers and identifiers of the file types that each application can provide. In one example, the application identifier can be the icon or name of an application that is already installed and can be run on the user terminal, and the identifier of the file types that each application can provide can be a brief description of the file types that each application can provide. For example, for a video application, the identifier of the file types that each application can provide can be short videos, movies, TV series, advertisements, and notification sounds, etc.

[0062] Step 2: The user terminal receives selection information provided by the user, which is generated by the user based on the application list. In one example, the user selects the aforementioned video application, as well as short videos, movies, TV series, and advertisements. In this case, the selection information includes the identifier of the video application and the identifiers of the file types such as short videos, movies, TV series, and advertisements. Through the aforementioned steps and scheme, the user can freely select one or more types of files from the applications that may be automatically muted. Therefore, the user can also freely select applications or one or more types of files from applications that may not be automatically muted. For example, the notification sounds played by applications generally do not cause discomfort to those around them, so the user can choose not to have the notification sounds automatically muted.

[0063] Step 3: The user terminal determines one or more applications that are running on the user terminal.

[0064] Step 4: The user terminal compares one or more running applications with the selection information to mark one or more applications;

[0065] Step 5: The user terminal receives one or more files; in one example, the user can send instructions to the server through the user terminal's application to request relevant content from the server. Subsequently, the user terminal can receive one or more of the content requested by the user terminal from the server, which is sent to the user terminal in the form of files.

[0066] Step 6: The user terminal selects one or more pre-selected files from one or more files to be used by one or more tagged applications. In one example, if the selection information includes the identifier of a video application, then the files to be used by that video application are all pre-selected files.

[0067] In a preferred embodiment, the data processing method of the present invention further includes:

[0068] The user terminal selects one or more tagged files from one or more pre-selected files based on selection information; in one example, if the selection information includes the identifier of a video application and the identifier of file types such as short videos, movies, TV series and advertisements, then all short video files received by the user terminal that will be played in the video application will be defined as tagged files, or in another case, all TV series files received by the user terminal that will be played in the video application will be defined as tagged files.

[0069] The user terminal receives operation instructions from the user, wherein the operation instructions indicate the application to be run, and the operation instructions also indicate a file to be run; in one example, the operation instructions are, for example, instructions to play audio or video (in this invention, video should be understood as a combination of audio and image, so playing video is equivalent to playing audio at the same time);

[0070] The user terminal determines whether the application indicated by the operation command is one or more marked applications;

[0071] If the user terminal determines that the application indicated by the operation command is one or more marked applications, then the user terminal continues to determine whether the file to be run indicated by the operation command is one or more marked files. In this invention, the reason why the user terminal first determines whether the application is one or more marked applications, and then determines whether the file to be run indicated by the operation command is one or more marked files, is that this operation mode can increase the user's flexibility in choosing whether audio will be automatically muted. For example, for the same file type (e.g., short video type), some applications offer elegant short videos, while others offer popular short videos. If the user is attending a high-end party, the user does not care whether the elegant short videos are automatically muted, but the user cannot allow popular short videos to be played at a high-end party. In this case, the method of this invention can achieve flexible control over whether the audio will be automatically muted for the same file type.

[0072] If the user terminal determines that the file to be run is one of one or more marked files, then the user terminal determines whether the user has enabled the silent mode. Those skilled in the art should understand that the silent mode of this invention differs from the silent mode of existing technologies. In existing technologies, once a mobile phone is in silent mode, the speaker will be turned off, and the audio will be muted regardless of what audio the user plays.

[0073] Example 3

[0074] In Example 3, the data processing method of the present invention further includes:

[0075] If the user terminal determines that the user has enabled the silent mode, then the user terminal will activate the ambient sound monitoring device set on the user terminal.

[0076] The ambient sound around the user terminal is monitored by an ambient sound monitoring device.

[0077] The ambient sound monitoring device determines whether the first ambient sound decibel level is higher than the ambient sound decibel threshold.

[0078] If the ambient sound monitoring device determines that the first ambient sound decibel level is higher than the ambient sound decibel threshold, the user terminal will turn off the ambient sound monitoring device and run a file based on the operation command. In one example, the ambient sound decibel threshold can be a decibel level greater than the maximum playback volume of the speaker. For instance, a typical radio broadcast produces a volume of around 40 decibels (data from Baidu Encyclopedia). The decibel level of a mobile phone speaker can be referenced to the radio playback volume, which is roughly 40 decibels. Therefore, the ambient sound decibel threshold can be set to 50 decibels (the noise level in a typical small restaurant is 50 decibels). In this case, if the ambient sound monitoring device determines that the first ambient sound decibel level is higher than the ambient sound decibel threshold, the user terminal can determine that the user is in a noisier environment than a small restaurant. In this situation, even when playing a video at maximum volume, the volume played by the mobile phone speaker is less than or even far less than the ambient noise. In this case, the user terminal can turn off the ambient sound monitoring device and run a file based on the operation command.

[0079] In a preferred embodiment, the data processing method further includes:

[0080] If the ambient sound monitoring device determines that the first ambient sound decibel level is lower than the ambient sound decibel threshold, the user terminal will continue to keep the ambient sound monitoring device running and will start running a file. Continuing with the previous example, if the ambient sound monitoring device determines that the first ambient sound decibel level is lower than the ambient sound decibel threshold, then the user terminal can determine that the user is in a relatively quiet environment. At this time, if a video is played at maximum volume, the volume played by the mobile phone speaker will be greater than the ambient noise. Therefore, any sound played by the user terminal at this time may cause discomfort to the user and others.

[0081] After the user terminal starts running a file, the ambient sound monitoring device continues to monitor the second ambient sound around the user terminal.

[0082] The ambient sound monitoring device determines whether the second ambient sound decibel level is higher than the forced silence decibel threshold. In one example, the second ambient sound decibel level is actually the superposition of ambient noise and the sound played by the user terminal's speaker; that is, the second ambient sound decibel level is contributed by the ambient noise and the sound played by the user terminal's speaker. In one example, the forced silence decibel threshold can be a multiple of the determined first ambient sound decibel level. For example, the forced silence decibel threshold can be equal to 1.05 times, 1.1 times, 1.2 times, etc., of the determined first ambient sound decibel level. For example, if the user terminal measures a first ambient sound decibel level of 30 decibels and a second ambient sound decibel level of 35 decibels, then the contribution of the user terminal's speaker to the ambient sound decibel level is equivalent to 5 decibels. The determination can be made through on-site testing (e.g., experimental testing by engineers from the software vendor). If, when the first ambient sound level is 30 decibels, the mobile phone's contribution to the ambient sound level is equivalent to 5 decibels, will the sound played by the user terminal speaker be noticeably noticed by others? If, when the first ambient sound level is 30 decibels, the user terminal speaker's contribution to the ambient sound level is equivalent to 5 decibels, the sound played by the user terminal speaker will be noticeably noticed by others. Therefore, the forced mute decibel threshold can be set to 1.05 times the determined first ambient sound level. Since the second ambient sound level is less than the forced mute decibel threshold, the mute component set in the user terminal will directly send a shutdown command to the user terminal's speaker component according to the following steps.

[0083] If the ambient sound monitoring device determines that the second ambient sound decibel level is higher than the forced silence decibel threshold, the mute component installed in the user terminal directly sends a shutdown command to the speaker assembly of the user terminal. This shutdown command from the mute component overrides the CPU's control commands on the speaker assembly. Those skilled in the art should understand that "override" is the English equivalent of "override," and while different translations may exist (e.g., covering, copying, etc.), the meaning is the same. In this invention, "override" means that the mute component's shutdown command enables the speaker to ignore the CPU's command and execute the mute component's shutdown command. For example, the CPU may have sent a play command to the speaker, and the speaker may be playing, but once the mute component sends a shutdown command, the speaker immediately shuts down, regardless of the CPU's previous play command. During this process, although the user terminal is playing a file with sound, the determination of whether the second ambient sound decibel level of the second ambient sound is higher than the forced mute decibel threshold by the ambient sound monitoring device and the direct sending of a shutdown command from the mute component within the user terminal to the speaker component of the user terminal can both be completed in a very short time. The mute component's only function is to send shutdown commands, so there is no problem of the speaker not shutting down in time due to high CPU usage. From the user's perspective, the file with sound may be automatically muted after 0.5 seconds of playback, which will not cause discomfort to the user or others.

[0084] Example 4

[0085] In Example 4, the data processing method of the present invention further includes:

[0086] If the user terminal determines that the user has turned off the silent mode, then the user terminal will turn off the ambient sound monitoring device and the mute component set on the user terminal.

[0087] After the user terminal disables the ambient sound monitoring device and mute component set on the user terminal, the user terminal starts running the file to be executed.

[0088] In a preferred embodiment, the data processing method further includes:

[0089] If the user terminal determines that the file to be run is not one of the one or more marked files, then the user terminal shall turn off the ambient sound monitoring device and the mute component set on the user terminal.

[0090] After the user terminal disables the ambient sound monitoring device and mute component set on the user terminal, the user terminal starts running the file to be executed.

[0091] In a preferred embodiment, the data processing method further includes:

[0092] The user terminal determines whether the user has downloaded a new application;

[0093] After a new application is installed on the user terminal, the user terminal sends an updated application list to the user. The updated application list only includes the identifier of the new application and the identifier of the file types that the new application can provide.

[0094] The user terminal receives update selection information provided by the user, which is generated by the user based on the list of applications to be updated.

[0095] The user terminal updates the selection information using the updated application list.

[0096] Example 5

[0097] The present invention provides a data processing apparatus, including a unit for performing the following operations:

[0098] The user terminal provides the user with a list of applications, which includes application identifiers and identifiers of the file types that each application can provide;

[0099] The user terminal receives selection information provided by the user, where the selection information is generated by the user based on a list of applications;

[0100] The user terminal determines one or more applications that are running on it.

[0101] The user terminal compares one or more running applications with the selection information to identify one or more applications;

[0102] One or more files are received by the user terminal;

[0103] The user terminal selects one or more pre-selected files from one or more files to be used by one or more tagged applications.

[0104] In a preferred embodiment, the data processing apparatus further includes a unit for performing the following operations:

[0105] The user terminal selects one or more tagged files from one or more pre-selected files based on selection information.

[0106] The user terminal receives operation instructions from the user, wherein the operation instructions instruct the application to be run, and the operation instructions also instruct a file to be run;

[0107] The user terminal determines whether the application indicated by the operation command is one or more marked applications;

[0108] If the user terminal determines that the application indicated by the operation command is one or more marked applications, the user terminal shall continue to determine whether the file to be run indicated by the operation command is one or more marked files.

[0109] If the user terminal determines that the file to be run is one of one or more marked files, then the user terminal determines whether the user has enabled the silent mode.

[0110] The present invention provides a data processing system, including a processor and a memory, wherein the memory stores computer-executable instructions, which, when executed by the processor, cause the system to perform the method described above.

[0111] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A data processing method, characterized in that, The data processing method includes: The user terminal provides the user with a list of applications, wherein the list of applications includes application identifiers and identifiers of the file types that each application can provide; The user terminal receives selection information provided by the user, wherein the selection information is generated by the user based on the application list; The user terminal determines one or more applications that are running on the user terminal. The user terminal compares the one or more running applications with the selection information to mark one or more applications; One or more files are received by the user terminal; The user terminal selects one or more pre-selected files from the one or more files to be used by the one or more tagged applications. The data processing method further includes: The user terminal selects one or more tagged files from the one or more pre-selected files based on the selection information; The user terminal receives an operation instruction from the user, wherein the operation instruction indicates the application to be run, and the operation instruction also indicates a file to be run; The user terminal determines whether the application indicated by the operation command is one or more marked applications; If the user terminal determines that the application indicated by the operation instruction is one or more marked applications, the user terminal will continue to determine whether the file to be run indicated by the operation instruction is one or more marked files; If the user terminal determines that the file to be run is one of the one or more marked files, then the user terminal determines whether the user has enabled the silent mode. The data processing method further includes: If the user terminal determines that the user has enabled the silent mode of the user terminal, then the user terminal will activate the ambient sound monitoring device set on the user terminal. The ambient sound around the user terminal is monitored by an ambient sound monitoring device. The ambient sound monitoring device determines whether the first ambient sound decibel level of the first ambient sound is higher than the ambient sound decibel threshold. If the ambient sound monitoring device determines that the first ambient sound decibel level of the first ambient sound is higher than the ambient sound decibel threshold, then the user terminal will turn off the ambient sound monitoring device and run the file to be run based on the operation command. The data processing method further includes: If the ambient sound monitoring device determines that the first ambient sound decibel level of the first ambient sound is lower than the ambient sound decibel threshold, then the user terminal will continue to turn on the ambient sound monitoring device and start running the file to be run. After the user terminal starts running a file to be run, the ambient sound monitoring device continues to monitor the second ambient sound around the user terminal; The ambient sound monitoring device determines whether the second ambient sound decibel level is higher than the forced silence decibel threshold. If the ambient sound monitoring device determines that the second ambient sound decibel is higher than the forced silence decibel threshold, the mute component in the user terminal directly sends a shutdown command to the speaker component of the user terminal. If the user terminal determines that the user has turned off the mute mode of the user terminal, the user terminal turns off the ambient sound monitoring device and the mute component on the user terminal. After the user terminal disables the ambient sound monitoring device and the mute component on the user terminal, the user terminal starts running the file to be run.

2. The data processing method as described in claim 1, characterized in that, The shutdown command sent by the mute component can override the CPU's control commands to the speaker component in the user terminal. Overriding means that the shutdown command sent by the mute component can make the speaker ignore the commands sent by the CPU and execute the shutdown command sent by the mute component. The data processing refers to video or audio data processing.

Images