Audio control method, electronic pen and electronic equipment system

Abstract

The invention discloses an audio control method, an electronic pen and an electronic equipment system. The method comprises the following steps: obtaining touch input of a user; determining a touch feature corresponding to the touch input; based on the touch characteristics, adjusting a signal value of a currently output sample audio to respond to the touch input, the sample audio being an audio for simulating friction between a pen point and a writing surface; wherein for the first touch feature, the sample audio has a first target signal value; for the second touch feature, the sample audio has a second target signal value, the first target signal value is different from the second target signal value, and the first touch feature is different from the second touch feature.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an audio control method, an electronic pen, and an electronic device system. Background Technology

[0002] With the continuous development of electronic devices, it has become increasingly common for users to use their fingers or styluses to input handwriting on touchscreens. For example, users can use their fingers or styluses to input Chinese characters, symbols, lines (trajectories), or drawings on touchscreens.

[0003] However, writing on paper has become the habit of most users, and the process of handwriting input on a touch screen using a finger or stylus is rather monotonous and cannot present the interactive effect of writing on paper. Summary of the Invention

[0004] On the one hand, this application provides an audio control method, including:

[0005] Obtain user touch input;

[0006] Determine the touch features corresponding to the touch input;

[0007] Based on the touch features, the signal value of the currently output sample audio is adjusted to respond to the touch input, wherein the sample audio is used to simulate the friction between the pen tip and the writing surface;

[0008] Specifically, for the first touch feature, the sample audio has a first target signal value; for the second touch feature, the sample audio has a second target signal value, the first target signal value is different from the second target signal value, and the first touch feature is different from the second touch feature.

[0009] In one possible scenario, adjusting the signal value of the currently output sample audio includes:

[0010] Modify the amplitude of the currently output sample audio;

[0011] Modify the gain coefficient of the high-frequency components in the currently output sample audio, and

[0012] Modify the audio frequency of the currently output sample audio.

[0013] In yet another possible scenario, the touch feature includes touch pressure;

[0014] If the touch pressure of the touch input changes from the first touch pressure to the second touch pressure, modifying the amplitude of the currently output sample audio includes: increasing the scaling factor of the amplitude of the currently output sample audio so that the amplitude of the sample audio increases linearly.

[0015] Furthermore, modifying the gain coefficient of the high-frequency components in the currently output sample audio includes: controlling the gain coefficient of the high-frequency components in the currently output sample audio to increase in a stepwise manner.

[0016] The second touch pressure is greater than the first touch pressure.

[0017] In yet another possible scenario, the touch feature includes touch speed;

[0018] If the touch speed of the touch input changes from a first touch speed to a second touch speed, adjusting the audio frequency of the currently output sample audio includes: controlling the playback rate of the currently output sample audio to increase linearly;

[0019] Furthermore, adjusting the signal value of the currently output sample audio also includes: controlling the cutoff frequency of the high-frequency component to shift upward;

[0020] The second touch speed is greater than the first touch speed.

[0021] In another possible implementation, the touch features include touch pressure and touch speed, and the signal value for adjusting the currently output sample audio includes:

[0022] Based on the touch pressure and touch speed, determine the appropriate target amplitude for the currently output sample audio.

[0023] Based on the target amplitude, the signal value of the sample audio is adjusted to the target signal value, wherein the sample audio has the target amplitude when it has the target signal value.

[0024] In another possible scenario, determining the appropriate target amplitude for the currently output sample audio based on the touch pressure and touch speed includes:

[0025] Obtain the set basic amplitude and velocity sensitivity coefficient;

[0026] Based on the base amplitude, the speed sensitivity coefficient, the touch speed, and the pressure level corresponding to the touch pressure, the target amplitude suitable for the currently output sample audio is determined.

[0027] In another possible scenario, adjusting the signal value of the currently output sample audio based on the touch features includes:

[0028] Based on the touch features, an intelligent model is used to determine the target signal value suitable for the currently output sample audio.

[0029] Adjust the signal value of the sample audio to the target signal value;

[0030] The intelligent model is trained based on touch feature samples corresponding to at least one touch input sample, with the training objective being that the predicted audio signal value matches the actual audio signal value corresponding to the touch input sample.

[0031] Furthermore, this application also provides an electronic pen, comprising:

[0032] A sensing module is used to respond to touch input and send the sensed touch features to the touch screen of an electronic device;

[0033] A receiving module is used to receive control commands fed back by the touch screen; wherein the control commands are generated by the electronic device based on the touch features and are used to control and adjust the signal value of the currently output sample audio, wherein the sample audio is audio used to simulate the friction between the pen tip and the writing surface;

[0034] The processing module is used to adjust the signal value of the currently output sample audio based on the control command;

[0035] An output interface is used to generate a vibration drive signal based on the adjusted signal value of the sample audio and to respond to the vibration drive signal.

[0036] In another aspect, this application also provides an electronic device system, comprising:

[0037] The sensing module, configured in the stylus or touchscreen, is used to capture the touch features of touch input;

[0038] An audio modification engine is used to adjust the signal value of the currently output sample audio based on the touch feature to respond to the touch input. The sample audio is audio used to simulate the friction between a pen tip and a writing surface. For a first touch feature, the sample audio has a first target signal value; for a second touch feature, the sample audio has a second target signal value. The first target signal value and the second target signal value are different, and the first touch feature and the second touch feature are different.

[0039] An output interface, configured in a stylus or touchscreen, is used to output the adjusted signal value of the sample audio.

[0040] In one possible scenario, the audio modification engine includes:

[0041] A sample library is used to store multiple sets of candidate sample audio for simulating basic paper friction, wherein the sample audio belongs to one of the multiple sets of candidate sample audio.

[0042] A parameter mapping unit is used to map the touch features to corresponding audio modification parameters, wherein the touch features include at least one of touch pressure and touch speed;

[0043] A real-time modulator modulates the byte values ​​of the sample audio based on audio modification parameters. Attached Figure Description

[0044] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0045] Figure 1 A flowchart illustrating the audio control method provided in this application;

[0046] Figure 2 Another flowchart illustrating the audio control method provided in this application;

[0047] Figure 3 Another flowchart illustrating the audio control method provided in this application;

[0048] Figure 4 A schematic diagram of the composition structure of an electronic pen provided in this application;

[0049] Figure 5 This is a schematic diagram of the composition structure of an electronic device system provided in this application. Detailed Implementation

[0050] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0051] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0052] like Figure 1 This illustration shows a flowchart of an audio control method provided in this application. The method of this embodiment can be applied to an electronic device, which can be any electronic device with a touch screen, such as a mobile phone, laptop, or tablet computer. The solution of this application can also be applied to an electronic device system including an electronic device and a stylus, wherein the electronic device has a touch screen.

[0053] The method in this embodiment may include:

[0054] S101, obtains the user's touch input.

[0055] For example, electronic devices receive touch input from users through their fingers or styluses (also known as electronic pens or styluses) via a touchscreen.

[0056] S102, determine the touch features corresponding to the touch input.

[0057] The touch features corresponding to the touch input are used to characterize the input characteristics of the touch input. For example, the input features can be input speed, input force, etc. Based on this, the touch features of the touch input can include at least one of the following features: touch pressure and touch speed.

[0058] Determining touch features can involve the sensing module in the touchscreen of an electronic device capturing the touch input characteristics. For example, the touch pressure can be obtained using a pressure sensor in the touchscreen. If the touch input is via a stylus, determining the touch features can also involve sensing the touch features using the stylus's sensing module and sending them to the electronic device's touchscreen.

[0059] It is understandable that when a user is drawing or inputting characters using a touchscreen, the user's touch input can be continuous over a period of time. Therefore, step S102 can be executed multiple times, such as once every set time interval, without any specific limitation.

[0060] S103 adjusts the signal value of the currently output sample audio based on touch characteristics to respond to touch input.

[0061] The sample audio is used to simulate the friction between the pen tip and the writing surface. For example, the sample audio can be the pre-collected and stored audio of the pen tip writing on paper, or audio obtained by superimposing noise with a sine wave, etc., without any specific restrictions.

[0062] In this application, if the electronic device senses touch input, it outputs sample audio. Therefore, as long as the user's touch input continues, the electronic device will continuously output sample audio. However, the sample audio output in this application is not fixed, but rather the signal value of the sample audio is adjusted based on the touch characteristics, so that the output sample audio dynamically changes with the changes in the touch characteristics of the touch input.

[0063] Unlike simply adjusting the volume of the sample audio, this application can change the audio signal information of the sample audio by adjusting the signal value of the currently output sample audio, thereby changing at least one of the semantic information (e.g., the meaning or emotion expressed by the sample audio) and non-semantic information (e.g., the timbre or rhythm of the sample audio) of the output sample audio, so that the output sample audio can match the touch features of the touch input currently obtained by the electronic device.

[0064] For example, by adjusting the signal value of the currently output sample audio, changing the spectral envelope and temporal structure of the sample audio, the audio content contained in the sample audio can be changed, thereby changing the semantic information expressed by the sample audio.

[0065] For example, adjusting the signal value of a sample audio signal can also be done by changing the amplitude and frequency of the sample audio without altering the semantic information expressed by the audio.

[0066] Specifically, for the first touch feature, the sample audio has a first target signal value; for the second touch feature, the sample audio has a second target signal value. The first target signal value and the second target signal value are different, and the first touch feature and the second touch feature are different. Therefore, when the touch input feature changes from the first touch feature to the second touch feature, the signal value of the sample audio currently output by the electronic device will also be adjusted from the first target signal value to the second target signal value, thus causing a change in the sample audio.

[0067] It is understood that when the solution of this application is applied to an electronic device or electronic device system, the electronic device can adjust the signal value of its currently output sample audio, thereby changing the currently output sample audio.

[0068] Specifically, when the solution of this application is applied to an electronic device system, a stylus can be used to generate audio to simulate the friction between the pen tip and the writing surface. Based on this, the electronic device can generate control commands based on touch features. These control commands are used to adjust the signal value of the currently output sample audio. Correspondingly, the stylus can adjust the signal value of the currently output sample audio based on the control commands, and generate and respond to a vibration drive signal based on the adjusted sample audio signal value. By responding to this vibration drive signal, the stylus's motor and other output units can vibrate, producing a vibration audio corresponding to the adjusted sample audio signal value.

[0069] As can be seen from the above, this application adjusts the signal value of the currently output sample audio based on the touch features corresponding to the touch input. When the touch input feature is a first touch feature, the sample audio has a first target signal value; and when the touch input feature is a second touch feature, the sample audio has a second target signal value. This allows the signal value of the currently output sample audio to change synchronously with the changes in the touch input feature. Furthermore, since the sample audio is used to simulate the friction between the pen tip and the writing surface, the solution of this application can dynamically adjust the sample audio simulating writing sounds based on the touch features of the input touch, thereby presenting users with an interactive experience of writing on real paper or other writing surfaces.

[0070] As can be understood, as mentioned above, there are multiple ways to implement the adjustment of the signal value of the sample audio signal in this application. The following examples illustrate several possible scenarios for adjusting the signal value of the sample audio signal.

[0071] For the first possible case of adjusting the signal value of the sample audio signal, please refer to... Figure 2 As shown. Figure 2 The diagram illustrates a flowchart of an audio control method provided in this application. The method in this embodiment may include:

[0072] S201, obtain the user's touch input.

[0073] S202, determine the touch feature corresponding to the touch input.

[0074] The above two steps can be referred to in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0075] S203, based on touch features, uses an intelligent model to determine the appropriate target signal value for the currently output sample audio.

[0076] Specifically, for the first touch feature, the intelligent model determines that the sample audio is suitable for a first target signal value; for the second touch feature, the intelligent model determines that the sample audio is suitable for a second target signal value. The first target signal value and the second target signal value are different, and the first touch feature and the second touch feature are different.

[0077] The intelligent model is trained based on the touch feature samples corresponding to at least one touch input sample, with the training objective being that the predicted audio signal value matches the actual audio signal value corresponding to the touch input sample.

[0078] Understandably, for ease of distinction, the touch features of the touch input used as training samples are referred to as touch feature samples of the touch input samples. The specific types of features included in the touch feature samples of the touch input samples can be similar to the types of touch features of the touch input. For example, touch feature samples may include at least one of touch pressure and touch speed. For ease of distinction, the touch pressure and touch speed included in the touch feature samples can be referred to as touch pressure samples and touch speed samples, respectively.

[0079] In this application, the intelligent model can be a large language model, a multimodal large language model, or a deep learning model, or a combination of multiple models, etc., without any specific limitations.

[0080] The intelligent model can be trained using supervised training methods, and there are no restrictions on the specific training methods.

[0081] For example, consider touch input samples, including touch pressure samples and touch speed samples. After obtaining at least one touch input sample's corresponding touch pressure sample and touch speed sample, as well as the actual audio signal value (i.e., the labeled audio signal value) corresponding to each touch input sample, for each touch input sample, based on the corresponding touch pressure sample and touch speed sample, the intelligent model determines the predicted audio signal value corresponding to that touch input sample. Then, based on the predicted audio signal value and labeled audio signal value corresponding to each touch input sample, combined with the set loss function, the loss function value is calculated. If the loss function value has not yet converged or the number of iterations has not reached the set number, the parameters of the intelligent model are adjusted, and the operation of determining the predicted audio signal value corresponding to the touch input sample using the intelligent model is returned until the loss function value converges or the number of iterations reaches the set number, at which point the training of the intelligent model is considered complete.

[0082] Understandably, in practical applications, in order to more accurately determine the touch characteristics of the touch input and accurately determine the audio suitable for those touch characteristics, the touch characteristics corresponding to each touch input obtained in this application may include: touch characteristics of the touch input at at least one time point, such as touch pressure and touch speed at at least one time point. Based on this, the touch characteristics of the touch input at at least one time point can be input into the intelligent model to obtain the target signal value suitable for the currently output sample audio.

[0083] S204, adjust the signal value of the sample audio to the target signal value in response to the touch input.

[0084] The sample audio is used to simulate the friction between the pen tip and the writing surface.

[0085] Step S204 can be found in the relevant description of the previous embodiments, and will not be repeated here.

[0086] It is understandable that, considering the signal value of an audio signal is stored in binary form (i.e., byte value), electronic devices store the byte value of the audio signal. Based on this, one example of adjusting the signal value of the sample audio in this application is adjusting the byte value of the sample audio. Building upon this, in Figure 2 In this embodiment, determining the target signal value of the sample audio using a smart model can be equivalent to determining the target byte value of the sample audio. Correspondingly, the byte values ​​of the sample audio can be adjusted to the target byte value.

[0087] It's understandable that the currently output sample audio can be buffered, and a certain duration of sample audio will be buffered each time to ensure continuous output. Therefore, the currently output sample audio can correspond to multiple bytes. Based on this, adjusting the signal value of the sample audio can mean adjusting the value of each byte of the sample audio. The target byte value can be different at different time points. Therefore, the byte value of the sample audio at each time point can be adjusted to the target byte value at that corresponding time point.

[0088] For ease of distinction, the touch features at at least one time point corresponding to the touch input are referred to as the touch features at at least one first time point. The target byte value at at least one second time point in the sample audio is referred to as the target byte value at at least one second time point. Based on this, an intelligent model can be used to determine the target byte value at at least one second time point of the sample audio.

[0089] In this embodiment, based on the touch features of the touch input, a trained intelligent model can be used to determine the target signal value suitable for the currently output sample audio. By leveraging the predictive accuracy of the intelligent model, the target signal value of the sample audio that matches the current touch features of the touch input can be determined more reasonably and accurately. This allows for more reasonable control and adjustment of the output sample audio, thereby providing users with the experience of writing on paper or other writing surfaces during touch input such as writing or drawing on the touch screen.

[0090] In the second possible scenario, adjusting the signal value of the sample audio can be achieved by adjusting the signal value in the following ways:

[0091] Modify the amplitude of the currently output sample audio;

[0092] Modify the gain coefficient of the high-frequency components in the currently output sample audio;

[0093] Additionally, it modifies the audio frequency of the currently output sample audio.

[0094] Modifying the amplitude of the sample audio is essentially changing the intensity of the corresponding sound wave, which in turn changes the loudness of the sample audio.

[0095] In this context, the gain coefficient of the high-frequency components in the sample audio is the amplification factor of the amplitude of the high-frequency signal (e.g., typically >2kHz) in the sample audio. It affects the brightness, clarity, and spatiality of the sample audio, making it a key parameter influencing the sound quality. Therefore, by modifying the gain coefficient of the high-frequency components in the sample audio, the brightness and spatiality of the sample audio can be improved.

[0096] The audio frequency of the sample audio can be either the playback frequency of the sample audio or the audio frequency of the sample audio itself. Modifying the playback frequency of the sample audio can be done by changing the number of audio frames or bytes of the sample audio output per unit time, thereby adjusting the playback rate of the sample audio.

[0097] The audio frequency of the sample audio reflects the speed of sound wave vibration. By modifying the signal value of the sample audio, the speed of sound wave vibration can be changed, thereby changing the audio frequency of the sample audio.

[0098] The specific methods for modifying the amplitude of the sample audio, the gain coefficient of the high-frequency components, and the audio frequency are related to the changes in the touch pressure of the input features. Several possible implementation methods are illustrated below:

[0099] In one possible implementation, the touch input features include touch pressure, such as sensing the touch input pressure via a pressure sensor on a touchscreen or stylus. In this case, if the touch input pressure changes from a first touch pressure to a second touch pressure, modifying the amplitude of the currently output sample audio can include increasing the scaling factor of the amplitude of the currently output sample audio so that the amplitude of the sample audio increases linearly. Correspondingly, if the touch input pressure changes from a first touch pressure to a second touch pressure, modifying the gain coefficient of the high-frequency components in the currently output sample audio can include controlling the gain coefficient of the high-frequency components in the currently output sample audio to increase in a stepwise manner. Wherein, the second touch pressure is greater than the first touch pressure.

[0100] Similarly, if the touch input pressure changes from the second touch pressure to the first touch pressure, the scaling factor of the amplitude of the currently output sample audio can be reduced, and the gain coefficient of the high-frequency components in the sample audio can be reduced stepwise.

[0101] The gain coefficient of the high-frequency component of the sample audio is increased or decreased in a stepwise manner, which can increase or decrease the target step value of the high-frequency component of the sample audio. The size of the target step value can be set according to actual needs, or it can be determined according to the difference between the first touch pressure and the second touch pressure, without any restrictions.

[0102] In another possible implementation, the touch feature includes touch speed. If the touch speed of the touch input changes from a first touch speed to a second touch speed, the playback rate of the currently output sample audio can be controlled to increase linearly. The second touch speed is greater than the first touch speed.

[0103] For example, there is a linear relationship between the playback rate of the sample audio and the touch speed or the change in touch speed. Based on this, if the touch speed of the touch feature increases, the playback rate of the sample audio will also increase linearly. Of course, there are other possibilities for how to linearly increase the playback rate of the sample audio when the touch speed of the touch input increases, and there are no restrictions on this.

[0104] Furthermore, to improve the audio quality of the sample audio, in this application, if the touch input speed changes from a first touch speed to a second touch speed, adjusting the signal value of the currently output sample audio may further include: shifting the cutoff frequency of the high-frequency components of the sample audio upwards. Shifting the cutoff frequency of the high-frequency components of the sample audio upwards can optimize the listening experience of the sample audio, reduce noise, and increase the effective information density in the sample audio, thereby improving the audio quality of the output sample audio.

[0105] Correspondingly, if the touch speed of the sample audio changes from the second touch speed to the first touch speed, the playback rate of the currently output sample audio can be controlled to decrease linearly. Furthermore, the cutoff frequency of the high-frequency components of the sample audio can be further controlled to shift downwards.

[0106] There are multiple ways to determine the touch speed of touch input, and there are no specific restrictions.

[0107] For example, in one possible implementation, the touch coordinates of the touch input at at least one time point (i.e., at least one first time point) can be obtained, the touch speed of the touch input at the at least one time point can be determined based on the touch coordinates at the at least one time point, and the average value of the touch speed at the at least one time point can be determined as the touch speed of the touch input.

[0108] For example, touch input at a point in time touch speed It can be calculated using the following formula:

[0109] (Formula 1);

[0110] in, The touch input at different times x and y coordinates For touch input at time points The previous time point The x-coordinate and y-coordinate of the time.

[0111] Specifically, after calculating the touch speed at at least one time point, this application can further smooth the touch speed at that time point using methods such as sliding window mean filtering or Kalman filtering to reduce noise interference. Based on this, the touch speed of the touch input can be determined according to the smoothed touch speed at each time point.

[0112] In another possible implementation, if the touch input speed changes from a first touch speed to a second touch speed, this application can further control the audio frequency of the currently output sample audio to increase linearly; conversely, if the touch input speed changes from a second touch speed to a first touch speed, the audio frequency of the currently output sample audio can be controlled to decrease linearly. By increasing the audio frequency of the sample audio, the pitch of the sample audio can be increased, making the sample audio sharper and reducing the timbre's fullness; conversely, by decreasing the audio frequency of the sample audio, the pitch of the sample audio can be decreased, making the sample audio deeper and increasing the timbre's fullness.

[0113] It is understandable that the above is an introduction to the specific implementation of adjusting the signal value of sample audio, taking touch input features, including touch speed or touch pressure, as examples. In practical applications, touch input features can include both touch speed and touch pressure. Based on this, this application can combine the two implementation methods mentioned above to comprehensively adjust the amplitude, audio frequency, and playback rate of the sample audio, so as to adjust the signal value of the sample audio.

[0114] In the second possible scenario of adjusting the signal value of the sample audio, considering that the sample audio is stored in bytes, modifying the signal value of the sample audio is also equivalent to modifying the byte value of the sample audio.

[0115] For example, modifying the amplitude of a sample audio can be done by increasing or decreasing each byte value of the sample audio by the same factor, thereby increasing or decreasing the gain coefficient of the sample audio amplitude, and thus increasing or decreasing the amplitude of the sample audio by the corresponding factor. The factor by which the byte values ​​are increased or decreased is related to the amount by which the touch speed is increased or decreased.

[0116] Similarly, modifying the playback rate of sample audio can be achieved by changing the number of bytes of sample audio output per unit time. Modifying the audio frequency of the sample audio itself, as well as the gain coefficient of high-frequency components, can also be achieved by adjusting the byte values ​​of the sample audio; details will not be elaborated further.

[0117] In the third possible scenario, the touch characteristics may include touch pressure and touch speed. In this case, adjusting the signal value of the currently output sample audio can be achieved by: determining a suitable target amplitude for the currently output sample audio based on the touch pressure and touch speed; and adjusting the signal value of the sample audio to the target signal value based on the target amplitude. Wherein, when the sample audio has the target signal value, the sample audio has the target amplitude.

[0118] There are several possible implementations for determining the appropriate target amplitude for the sample audio based on touch pressure and touch speed, and no restrictions are placed on this. The following explanation uses one implementation method as an example. Figure 3 This illustrates another flowchart of the audio control method provided in this application. The method in this embodiment may include:

[0119] S301 obtains the user's touch input.

[0120] S302, determine the touch pressure and touch speed corresponding to the touch input.

[0121] The specific implementation of determining touch pressure and touch speed can be found in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0122] S303, obtains the set basic amplitude and velocity sensitivity coefficient.

[0123] The basic amplitude is a pre-set amplitude value used as a reference standard.

[0124] The speed sensitivity coefficient can be used to limit the impact of touch speed on amplitude, and can be set according to actual needs.

[0125] S304 determines the target amplitude suitable for the currently output sample audio based on the base amplitude, speed sensitivity coefficient, touch speed, and pressure level corresponding to the touch pressure.

[0126] The touch pressure level can be determined based on the corresponding touch pressure range for different pressure levels. For example, the touch pressure levels can be divided into three types: light pressure, medium pressure, and high pressure. Each pressure level corresponds to a touch pressure range. For instance, a touch pressure greater than or equal to 0.7 is considered a high pressure level; a touch pressure greater than or equal to 0.3 but less than 0.7 is considered a medium pressure level; and a touch pressure less than 0.3 is considered a light pressure level.

[0127] In this embodiment, in addition to touch pressure and touch speed, a suitable target amplitude is determined by combining the basic amplitude and speed sensitivity coefficient. This helps to more reasonably determine the suitable target amplitude for the currently output sample audio. The specific implementation of determining this target amplitude is not limited.

[0128] For example, different pressure levels can correspond to different amplitude coefficients. For instance, when the pressure level of the touch control is high, the amplitude coefficient can be 1.0; when the pressure level of the touch control is medium, the amplitude coefficient can be 0.6; and when the pressure level of the touch control is light, the amplitude coefficient can be 0.3.

[0129] Based on this, the target amplitude suitable for the currently output sample audio can be determined by considering the base amplitude, speed sensitivity coefficient, touch speed, and the amplitude coefficient corresponding to the pressure level of the touch pressure. For example, the target amplitude is positively correlated with the touch speed, and the amplitude of change in the sample audio can be adjusted by using the speed sensitivity coefficient.

[0130] For example, target amplitude It can be calculated using the following formula:

[0131] (Formula 2);

[0132] in, Based on amplitude, For speed sensitivity coefficient, For touch speed, This is the amplitude coefficient corresponding to the pressure level of the touch control.

[0133] As can be seen from Formula 2 above, the amplitude of the sample audio can be controlled to change non-linearly with the touch input speed through the speed sensitivity coefficient, so as to simulate the effect of "fast writing with light sound and slow writing with heavy sound" in real writing, thereby simulating the real feeling of the user writing on a real writing surface.

[0134] S305, based on the target amplitude, adjusts the signal value of the sample audio to the target signal value in response to the touch input.

[0135] The sample audio is used to simulate the friction between the pen tip and the writing surface.

[0136] Specifically, when the sample audio has a target signal value, the sample audio has that target amplitude.

[0137] Specifically, for the first touch feature, the sample audio has a first target signal value; for the second touch feature, the sample audio has a second target signal value. The first target signal value is different from the second target signal value, and the first touch feature is different from the second touch feature.

[0138] The first touch feature may include a first touch pressure and a first touch speed, the second touch feature may include a second touch pressure and a second touch speed, and the difference between the first touch feature and the second touch feature may include at least one of the following: the first touch pressure and the second touch pressure are different, and the first touch speed and the second touch speed are different.

[0139] The specific implementation of adjusting the signal value of the sample audio to the target signal value can be found in the relevant descriptions of the previous embodiments, and will not be repeated here. For example, the byte value of the sample audio is adjusted to the target byte value corresponding to the target signal value. If the sample audio is stereo, the byte value of the sample audio corresponding to each channel can be adjusted separately, and the details will not be elaborated further.

[0140] It is understood that, in any of the above embodiments of this application, before obtaining touch input, this application may further determine the type of writing surface to be simulated; based on the writing surface type, determine the sample audio to be output. On this basis, the sample audio can be output in response to touch input.

[0141] This application allows for the pre-configuration of multiple selectable writing surface types. Based on the user's selection of the desired writing surface type in the configuration interface, the writing surface type to be simulated is determined. The selectable writing surface types include, but are not limited to, rough paper, smooth paper, and Xuan paper, etc., without specific restrictions.

[0142] Different types of writing surfaces correspond to different types of sample audio. The sample audio corresponding to the writing surface type is used to simulate the audio produced when the pen tip writes on that type of writing surface.

[0143] On the other hand, this application provides an electronic pen, such as Figure 4 This diagram illustrates one possible structural composition of the electronic pen provided in this application. Figure 4 It can be seen that the electronic pen includes at least:

[0144] The sensing module 401 is used to respond to touch input and send the sensed touch features to the touch screen of the electronic device;

[0145] The receiving module 402 is used to receive the control command fed back by the touch screen; wherein, the control command is generated by the electronic device based on the touch features, and is used to control and adjust the signal value of the currently output sample audio, the sample audio being the audio used to simulate the friction between the pen tip and the writing surface;

[0146] Processing module 403 is used to adjust the signal value of the currently output sample audio based on the control command;

[0147] Output interface 404 is used to generate a vibration drive signal based on the adjusted signal value of the sample audio and to respond to the vibration drive signal.

[0148] The touch features sensed by the sensing module may include at least one of touch input speed and touch pressure, as described above.

[0149] For example, the sensing module may include a pressure sensor to obtain the touch pressure. The sensing module may also include an accelerometer to sense the touch speed of the touch input, or the sensing module may determine the touch speed based on data such as the measurement start coordinates fed back from the touchscreen of the electronic device.

[0150] The control command is used to indicate the target signal value that the sample audio needs to be adjusted to. The specific implementation of the electronic device determining the target signal value that the sample audio needs to be adjusted to can be found in the previous embodiment of the specific implementation of adjusting the signal value of the electronic device, which will not be repeated here.

[0151] In this application, the output interface of the electronic pen may include a vibration device such as a motor, and the vibration device vibrates to produce the same sound effect as the sample audio. Therefore, the output interface needs to determine the vibration drive signal based on the signal value of the adjusted sample audio and respond to the vibration drive signal to drive the vibration device such as the motor to vibrate and produce a sound effect that matches the sample audio.

[0152] As can be seen, after the electronic pen's sensing module sends the sensed touch input features to the electronic device's touchscreen, the electronic device sends a control command, which determines the signal value of the sample audio used to control the current output, to the electronic pen's receiving module. Based on this, the electronic pen determines and responds to the vibration drive signal, using the signal value required by the control command sample audio and the adjusted sample audio signal value. This drives a motor and other vibration devices to vibrate and produce a sound effect matching the sample audio, allowing the audio generated by the electronic pen's output interface to change synchronously with the changes in the touch input features. Furthermore, since the electronic pen can generate a sound effect matching the sample audio by responding to the vibration drive signal, and the sample audio is used to simulate the friction between the pen tip and the writing surface, the solution of this application can dynamically adjust the audio of the electronic pen's simulated writing sound based on the touch input features, thus presenting users with an interactive experience of writing on a real paper or other writing surface.

[0153] Furthermore, this application also provides an electronic device system. For example... Figure 5 This application provides a schematic diagram of the composition of an electronic device system, which includes:

[0154] The sensing module 502, configured in a stylus or touchscreen, is used to capture the touch features of touch input;

[0155] An audio modification engine 502 is used to adjust the signal value of the currently output sample audio based on the touch feature to respond to the touch input. The sample audio is audio used to simulate the friction between a pen tip and a writing surface. For a first touch feature, the sample audio has a first target signal value; for a second touch feature, the sample audio has a second target signal value. The first target signal value and the second target signal value are different, and the first touch feature and the second touch feature are different.

[0156] Output interface 503, configured in a stylus or touch screen, is used to output the adjusted signal value of the sample audio.

[0157] The stylus can be any of the aforementioned styluses or electronic pens, or any electronic device capable of inputting into a touchscreen; there are no specific restrictions.

[0158] The sensing module may include a pressure sensor in a stylus or touchscreen to obtain the touch pressure of the touch input.

[0159] The sensing module may also include: a coordinate sensor in the touchscreen to obtain the touch coordinates of the touch input and determine the touch speed. Alternatively, the sensing module may include an accelerometer configured in the stylus or touchscreen to sense the touch speed of the touch input.

[0160] Of course, the sensor module can also include the pressure sensor, coordinate sensor and acceleration sensor mentioned above.

[0161] The adjusted signal value of the sample audio can be the sample audio with adjusted high-frequency component gain coefficient, audio frequency, and amplitude, as mentioned in the previous method embodiments. For details, please refer to the previous related introduction, which will not be repeated here.

[0162] For details on how this output interface is configured in a stylus, please refer to the previous introduction on the output interface of electronic pens; it will not be repeated here.

[0163] When the output interface is located on the touch screen of an electronic device, the output interface can be an audio output module such as a speaker.

[0164] The output interface may also include a vibration device such as a motor configured in a stylus and an audio output module configured in a touch screen of an electronic device. In this case, the signal value of the adjusted sample audio can be a composite signal of the audio from the vibration device and the audio output by the audio output module.

[0165] like Figure 5 As shown, this audio editing engine can be deployed in electronic devices.

[0166] exist Figure 5 To make it easier to understand, we will take the example of configuring the sensor module in the stylus and configuring the output interface in the touch screen.

[0167] The effects that this electronic device system can achieve can be found in the relevant descriptions of the preceding method embodiments, and will not be repeated here.

[0168] In one possible implementation, the audio modification engine includes:

[0169] A sample library is used to store multiple sets of candidate sample audio for simulating basic paper friction, wherein the sample audio belongs to one of the multiple sets of candidate sample audio.

[0170] A parameter mapping unit is used to map the touch features to corresponding audio modification parameters, wherein the touch features include at least one of touch pressure and touch speed;

[0171] A real-time modulator modulates the byte values ​​of the sample audio based on audio modification parameters.

[0172] The specific implementation of the real-time modulator modifying the byte values ​​of the sample audio can be found in the previous method embodiment's description of modifying the signal values ​​of the sample audio, and will not be repeated here.

[0173] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the audio control methods provided in this application.

[0174] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the audio control methods provided in this application.

[0175] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0177] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0178] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An audio control method, comprising: Obtain user touch input; Determine the touch features corresponding to the touch input; Based on the touch features, the signal value of the currently output sample audio is adjusted to respond to the touch input, wherein the sample audio is used to simulate the friction between the pen tip and the writing surface; Specifically, for the first touch feature, the sample audio has a first target signal value; for the second touch feature, the sample audio has a second target signal value, the first target signal value is different from the second target signal value, and the first touch feature is different from the second touch feature.

2. The audio control method according to claim 1, wherein adjusting the signal value of the currently output sample audio includes: Modify the amplitude of the currently output sample audio; Modify the gain coefficient of the high-frequency components in the currently output sample audio, and Modify the audio frequency of the currently output sample audio.

3. The audio control method according to claim 2, wherein the touch feature includes touch pressure; If the touch pressure of the touch input changes from the first touch pressure to the second touch pressure, modifying the amplitude of the currently output sample audio includes: increasing the scaling factor of the amplitude of the currently output sample audio so that the amplitude of the sample audio increases linearly. Furthermore, modifying the gain coefficient of the high-frequency components in the currently output sample audio includes: controlling the gain coefficient of the high-frequency components in the currently output sample audio to increase in a stepwise manner. The second touch pressure is greater than the first touch pressure.

4. The audio control method according to claim 2, wherein the touch feature includes touch speed; If the touch speed of the touch input changes from a first touch speed to a second touch speed, adjusting the audio frequency of the currently output sample audio includes: controlling the playback rate of the currently output sample audio to increase linearly; Furthermore, adjusting the signal value of the currently output sample audio also includes: controlling the cutoff frequency of the high-frequency component to shift upward; The second touch speed is greater than the first touch speed.

5. The audio control method according to claim 1, wherein the touch features include touch pressure and touch speed, and the signal value for adjusting the currently output sample audio includes: Based on the touch pressure and touch speed, determine the appropriate target amplitude for the currently output sample audio. Based on the target amplitude, the signal value of the sample audio is adjusted to the target signal value, wherein the sample audio has the target amplitude when it has the target signal value.

6. The audio control method according to claim 5, wherein determining the suitable target amplitude of the currently output sample audio based on the touch pressure and touch speed includes: Obtain the set basic amplitude and velocity sensitivity coefficient; Based on the base amplitude, the speed sensitivity coefficient, the touch speed, and the pressure level corresponding to the touch pressure, the target amplitude suitable for the currently output sample audio is determined.

7. The audio control method according to claim 1, wherein adjusting the signal value of the currently output sample audio based on the touch feature includes: Based on the touch features, an intelligent model is used to determine the target signal value suitable for the currently output sample audio. Adjust the signal value of the sample audio to the target signal value; The intelligent model is trained based on touch feature samples corresponding to at least one touch input sample, with the training objective being that the predicted audio signal value matches the actual audio signal value corresponding to the touch input sample.

8. An electronic pen: comprising: A sensing module is used to respond to touch input and send the sensed touch features to the touch screen of an electronic device; A receiving module is used to receive control commands fed back by the touch screen; wherein the control commands are generated by the electronic device based on the touch features and are used to control and adjust the signal value of the currently output sample audio, wherein the sample audio is audio used to simulate the friction between the pen tip and the writing surface; The processing module is used to adjust the signal value of the currently output sample audio based on the control command; An output interface is used to generate a vibration drive signal based on the adjusted signal value of the sample audio and to respond to the vibration drive signal.

9. An electronic device system, comprising: The sensing module, configured in the stylus or touchscreen, is used to capture the touch features of touch input; An audio modification engine is used to adjust the signal value of the currently output sample audio based on the touch feature to respond to the touch input. The sample audio is audio used to simulate the friction between a pen tip and a writing surface. For a first touch feature, the sample audio has a first target signal value; for a second touch feature, the sample audio has a second target signal value. The first target signal value and the second target signal value are different, and the first touch feature and the second touch feature are different. An output interface, configured in a stylus or touchscreen, is used to output the adjusted signal value of the sample audio.

10. The electronic device system of claim 9, wherein the audio modification engine comprises: A sample library is used to store multiple sets of candidate sample audio for simulating basic paper friction, wherein the sample audio belongs to one of the multiple sets of candidate sample audio. A parameter mapping unit is used to map the touch features to corresponding audio modification parameters, wherein the touch features include at least one of touch pressure and touch speed; A real-time modulator modulates the byte values ​​of the sample audio based on audio modification parameters.

Images