Vibration control method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a vibration control method and device, electronic equipment and a readable storage medium, and relates to the technical field of mobile phones, the vibration control method and device are applied to a processor of the electronic equipment, the electronic equipment further comprises a vibration motor, the processor is connected with the vibration motor, and the method comprises the steps that an ultrasonic signal is transmitted through a loudspeaker of the electronic equipment; under the condition that the finger of the user interacts with the display screen, a reflected ultrasonic signal is obtained through a microphone of the electronic equipment; based on a pre-trained finger position identification model, identifying a position area where the finger of the user currently interacts with the display screen of the electronic equipment through the reflected ultrasonic signal; and under the condition that the position area is the preset area, the vibration motor is controlled to work. The area where the finger of the user is located is analyzed and detected in real time through the ultrasonic signal, the vibration motor is controlled to work under the condition that the area where the finger interacts with the display screen is detected to be the preset area, and the vibration delay is low.

Description

Technical Field

[0001] The present application relates to the field of mobile phone technology, and more specifically, to a vibration control method, device, electronic device and readable storage medium. Background Art

[0002] At present, with the development of electronic information technology, users can interact with electronic devices, so that electronic devices can generate vibrations based on interactive operations to provide users with a better experience. However, currently, the delay of controlling the generation of vibrations based on interactive operations is relatively large. Summary of the invention

[0003] The present application proposes a vibration control method, device, electronic device and readable storage medium.

[0004] In a first aspect, an embodiment of the present application provides a vibration control method, which is applied to a processor of an electronic device, wherein the electronic device also includes a vibration motor, and the processor is connected to the vibration motor. The method includes: transmitting an ultrasonic signal through a speaker of the electronic device; when a user's finger interacts with a display screen, acquiring a reflected ultrasonic signal through a microphone of the electronic device; based on a pre-trained finger position recognition model, identifying a position area where the user's finger is currently interacting with the display screen of the electronic device through the reflected ultrasonic signal; and when the position area is a preset area, controlling the vibration motor to operate.

[0005] In the second aspect, the embodiment of the present application also provides a vibration control device, which is applied to a processor of an electronic device, wherein the electronic device also includes a vibration motor, the processor is connected to the vibration motor, and the device includes: a transmitting unit, an acquisition unit, a recognition unit, and a control unit. Among them, the transmitting unit is used to transmit an ultrasonic signal through the speaker of the electronic device; the acquisition unit is used to acquire the reflected ultrasonic signal through the microphone of the electronic device when the user's finger interacts with the display screen; the recognition unit is used to identify the position area where the user's finger is currently interacting with the display screen of the electronic device through the reflected ultrasonic signal based on a pre-trained finger position recognition model; the control unit is used to control the vibration motor to work when the position area is a preset area.

[0006] In a third aspect, an embodiment of the present application further provides an electronic device, a processor, a vibration motor, a speaker and a microphone, wherein the processor is respectively connected to the vibration motor, the microphone and the speaker, and the processor is used to execute the method described in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the method described in the first aspect above.

[0008] The vibration control method, device, electronic device and readable storage medium provided in the embodiments of the present application first transmit an ultrasonic signal through the speaker of the electronic device; then, when the user's finger interacts with the display screen, the reflected ultrasonic signal is obtained through the microphone of the electronic device; then, based on the pre-trained finger position recognition model, the position area where the user's finger is currently interacting with the display screen of the electronic device is identified through the reflected ultrasonic signal; thereby, when the position area is a preset area, the vibration motor is controlled to work. Since it is not necessary for the user to have interacted with the display screen in order to control the vibration motor to work according to the signal generated by the interaction. Instead, the area where the user's finger is located can be directly analyzed and detected in real time through the reflected ultrasonic signal. When it is detected that the area where the finger interacts with the display screen is a preset area, the vibration motor is controlled to work, with a lower vibration delay.

[0009] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 An application scenario diagram of the vibration control method provided in an embodiment of the present application is shown;

[0012] Figure 2 A method flow chart of a vibration control method provided by an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram showing an ultrasonic signal provided by an embodiment of the present application that is not reflected by a finger;

[0014] Figure 4 A schematic diagram showing an ultrasonic signal reflected by a finger provided in an embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram showing an ultrasonic signal reflected by a finger provided in yet another embodiment of the present application is shown;

[0016] Figure 6 A method flow chart of a vibration control method provided by another embodiment of the present application is shown;

[0017] Figure 7 A method flow chart of a vibration control method provided by another embodiment of the present application is shown;

[0018] Figure 8 The structure block diagram of the vibration control device provided by the embodiment of the present application is shown;

[0019] Fig. 9 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0020] Fig.10 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0022] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0023] At present, with the development of electronic information technology, users can interact with electronic devices, so that electronic devices can generate vibrations based on interactive operations to provide users with a better experience. However, the delay of controlling vibrations based on interactive operations is currently large. How to reduce the delay of controlling vibrations is an urgent problem to be solved.

[0024] Currently, when the interaction between the user and the display screen is detected, a subsequent detection can be further performed, and when the subsequent detection meets the set conditions, the electronic device can be controlled to vibrate.

[0025] However, the inventors have found in their research that if subsequent detection is awakened by interaction between the user and the display screen, the delay in the subsequent control to generate vibration may be relatively high.

[0026] Therefore, in order to solve or partially solve the above problems, the present application provides a vibration control method, device, electronic device and readable storage medium.

[0027] See also Figure 1 , Figure 1 The application scenario diagram of the vibration control method provided by the embodiment of the present application is shown, namely, a vibration control scenario 100, which may include an electronic device 110 and a user 120. Figure 1 As can be seen from the vibration control scenario 100 , the user 120 holds the electronic device 110 and interacts with the display screen of the electronic device 110 through fingers to control the electronic device 110 to run an application.

[0028] Furthermore, the electronic device 110 is provided with a speaker 113 and a microphone 114. Thus, the electronic device 110 can detect the location area where the user's finger is currently interacting with the display screen of the electronic device in combination with the speaker 113 and the microphone 114, and then control the vibration of the electronic device. For detailed description, please refer to the subsequent method embodiments.

[0029] See also Figure 2 , Figure 2 A method flow chart of a vibration control method provided by an embodiment of the present application is shown. The vibration control method can be applied to Figure 1 The electronic device in the vibration control scenario shown can specifically use a processor of the electronic device as an execution subject for executing the vibration control method. The vibration control method can include steps S110 to S140.

[0030] Step S110: transmitting an ultrasonic signal through a speaker of the electronic device.

[0031] The electronic device may include a vibration motor, which may be connected to a processor of the electronic device, so that the vibration motor may be controlled by the processor to operate or not operate. The vibration motor may be a small vibration motor, which may be controlled to operate in certain circumstances to generate vibration.

[0032] It is understandable that the user runs an application through an electronic device, so that the user can interact with the display screen of the electronic device through a finger to control the running application. For example, the running application may be a game application, so that the user can touch the icon displayed on the display screen with a finger to trigger the game application to perform a task corresponding to the touched icon. Therefore, in some embodiments, it is possible to determine whether the vibration motor needs to be controlled to work based on the position area where the user's finger interacts with the display screen of the electronic device. Specifically, the position area where the user's finger interacts with the display screen of the electronic device can be detected by an ultrasonic signal.

[0033] First, the ultrasonic signal can be emitted by a speaker of the electronic device, wherein the speaker can be connected to a processor, so that the processor can control the speaker to emit the ultrasonic signal.

[0034] Optionally, the electronic device may further include an audio power amplifier, which may be connected to the processor and the speaker respectively. The processor sends an audio signal corresponding to the ultrasonic signal to the audio power amplifier, and the audio power amplifier amplifies the power of the audio signal and outputs it through the speaker.

[0035] The speaker may be a micro speaker.

[0036] Step S120: When the user's finger interacts with the display screen, the reflected ultrasonic signal is acquired through the microphone of the electronic device.

[0037] It is understandable that the ultrasonic signal may pass through the user's finger during transmission, and thus be reflected by the user's finger. Therefore, the reflected ultrasonic signal can be collected by the electronic device.

[0038] In some implementations, when a user's finger interacts with the display screen, a reflected ultrasonic signal may be acquired through a microphone of the electronic device.

[0039] Among them, the interaction between the user's finger and the display screen can be that the user performs a touch operation on a certain position area on the display screen; or the user performs an air gesture operation on a certain position area on the display screen, which is not specifically limited in the embodiments of the present application.

[0040] The electronic device may be provided with a microphone, and the microphone may be connected to the processor, so that after the reflected ultrasonic signal is acquired by the microphone, the reflected ultrasonic signal may be transmitted to the processor.

[0041] It is understandable that, since the ultrasonic signal is an analog signal, and the processor is generally used to process digital signals, after the reflected ultrasonic signal is obtained through the microphone, the ultrasonic signal can also be encoded. Exemplarily, the electronic device may also include a coder-decoder (CODEC), which is connected to the processor and the microphone, respectively. Thus, the reflected ultrasonic signal obtained by the microphone can be input into the codec, and the reflected ultrasonic signal is encoded by the codec to obtain the encoded signal corresponding to the reflected ultrasonic signal, and then the encoded signal is transmitted to the processor.

[0042] Step S130: Based on the pre-trained finger position recognition model, the position area where the user's finger is currently interacting with the display screen of the electronic device is identified through the reflected ultrasonic signal.

[0043] It is understandable that since the relative position of the speaker to the electronic device is relatively fixed, for example, the speaker is generally set near a corner of the electronic device, such as the lower right corner area when the electronic device is used in portrait mode. When the user's finger is in different positions, the ultrasonic signal emitted by the speaker is reflected, which causes the reflected ultrasonic signal collected by the microphone to change.

[0044] For example, see also Figures 3 to 5 ,in Figure 3 FIG. 1 is a schematic diagram showing an ultrasonic signal collected by a microphone without being reflected by a finger. Figure 3 The two-dimensional plane coordinate 300 shown includes an x-axis and a y-axis that are perpendicular to each other, wherein the x-axis is used to represent time and the y-axis is used to represent amplitude. Figure 3 The curve 301 in FIG. 3 can represent the ultrasonic signal that has not been reflected by the finger.

[0045] akin, Figure 4 FIG. 1 is a schematic diagram showing an ultrasonic signal collected by a microphone and reflected by a finger. Figure 4 The two-dimensional plane coordinate 400 shown includes an x-axis and a y-axis that are perpendicular to each other, wherein the x-axis is used to represent time and the y-axis is used to represent amplitude. Figure 4 The curve 401 in FIG. 4 can represent the ultrasonic signal reflected by the finger, and the spatial distance between the finger and the microphone is gradually decreasing.

[0046] akin, Figure 5 FIG. 1 is a schematic diagram showing an ultrasonic signal collected by a microphone and reflected by a finger. Figure 5 The two-dimensional plane coordinate 500 shown includes an x-axis and a y-axis that are perpendicular to each other, wherein the x-axis is used to represent time and the y-axis is used to represent amplitude. Figure 5 The curve 501 in FIG. 5 represents the ultrasonic signal reflected by the finger, and the spatial distance between the finger and the microphone is gradually increasing.

[0047] It can be seen that when the user's finger interacts with different position areas on the display screen of the electronic device, the amplitude and spectrum characteristics of the reflected ultrasonic signal will show characteristics corresponding to each position area. Therefore, the position area where the user's finger is currently interacting with the display screen of the electronic device can be determined by the reflected ultrasonic signal. Specifically, based on a pre-trained finger position recognition model, the position area where the user's finger is currently interacting with the display screen of the electronic device can be identified by the reflected ultrasonic signal.

[0048] Among them, the pre-trained finger position recognition model can pre-learn the ultrasonic signals corresponding to various position areas of the finger on the display screen, so that the position area where the user's finger is currently interacting with the display screen of the electronic device can be determined by the reflected ultrasonic signal. For a detailed introduction, please refer to the subsequent embodiments.

[0049] Step S140: When the position area is a preset area, control the vibration motor to operate.

[0050] Furthermore, after obtaining the current position area of ​​the user's finger, the position area can be judged. If the position area is a preset area, the vibration motor can be controlled to work.

[0051] The preset area may be pre-set. Exemplarily, the preset area may be set based on the display screen, for example, a certain designated position area of ​​the display screen is set as the preset area, and specifically, for example, the designated position area may be the area corresponding to the lower right corner of the display screen. Another exemplary method is to set the preset area corresponding to each application based on different applications, and specifically, for example, for a game application, the area corresponding to the lower right corner of the display screen may be set as the preset area; for a video application, the area corresponding to the upper left corner of the display screen may be set as the preset area.

[0052] It should be noted that the above description of the preset area is only an example and does not constitute a specific limitation on the embodiments of the present application.

[0053] In some embodiments, the position area being a preset area may include that the position area is completely included in the preset area. It may also include that more than a threshold ratio of position areas are included in the preset area. For example, the threshold ratio may be 80%. In the case that more than 80% of the position areas are included in the preset area, the vibration motor may be controlled to operate.

[0054] Controlling the vibration motor to work may include sending a control signal to the vibration motor through a processor, so that the vibration motor can work based on the control signal.

[0055] The vibration control method provided in the embodiment of the present application first transmits an ultrasonic signal through the speaker of the electronic device; then, when the user's finger interacts with the display screen, the reflected ultrasonic signal is obtained through the microphone of the electronic device; then, based on the pre-trained finger position recognition model, the position area where the user's finger is currently interacting with the display screen of the electronic device is identified through the reflected ultrasonic signal; thereby, when the position area is a preset area, the vibration motor is controlled to work. Since it is not necessary for the user to have interacted with the display screen in order to control the vibration motor to work according to the signal generated by the interaction. Instead, the area where the user's finger is located can be directly analyzed and detected in real time through the reflected ultrasonic signal. When it is detected that the area where the finger interacts with the display screen is the preset area, the vibration motor is controlled to work, with a lower vibration delay.

[0056] See also Figure 6 , Figure 6 A method flow chart of a vibration control method provided by an embodiment of the present application is shown. The vibration control method can be applied to Figure 1 The electronic device in the vibration control scenario shown can specifically use a processor of the electronic device as an execution subject for executing the vibration control method. The vibration control method can include steps S210 to S260.

[0057] Step S210: transmitting an ultrasonic signal through a speaker of the electronic device.

[0058] Step S220: When the user's finger interacts with the display screen, the reflected ultrasonic signal is acquired through the microphone of the electronic device.

[0059] Among them, step S210 and step S220 have been introduced in detail in the above embodiments and will not be repeated here.

[0060] Step S230: Analyze the designated parameters of the reflected ultrasonic signal through the finger position recognition model to obtain characteristic parameters corresponding to the ultrasonic signal.

[0061] In some embodiments, a plurality of reference parameters may be stored in the finger position recognition model, and each reference parameter corresponds to a reference position area. The reference parameters are used to characterize characteristic parameters of ultrasonic signals returned by the user's fingers at different reference positions. The reference position may be the position of a pre-calibrated display screen. For example, the user's finger may be pre-controlled to be at a reference position, and then the ultrasonic signal reflected back at this time may be obtained, and characteristic parameters of the reflected ultrasonic signal may be extracted to obtain the reference parameters. Then, the user's finger may be placed at another reference position and the above steps may be repeated to obtain a plurality of reference parameters and a reference position area corresponding to each reference parameter.

[0062] Thus, the finger position recognition model can be used to analyze the specified parameters of the reflected ultrasonic signal to obtain characteristic parameters corresponding to the ultrasonic signal. The specified parameters may include at least one of amplitude and spectrum distribution. Therefore, for some embodiments, step S230 may also include step S231 and step S232.

[0063] Step S231: extracting the amplitude corresponding to each cycle of the reflected ultrasonic signal through the finger position recognition model.

[0064] Step S232: taking each period and the amplitude corresponding to the period as characteristic parameters corresponding to the ultrasonic signal.

[0065] Taking the case where the specified parameters include amplitude as an example, the amplitude corresponding to each cycle of the reflected ultrasonic signal can be extracted through the finger position recognition model.

[0066] The finger position recognition model may be a pre-trained model, such as a deep learning neural network model. In some embodiments, the finger position recognition model may be stored in a memory of an electronic device, so that after the reflected ultrasonic signal is acquired through the microphone of the electronic device, the processor of the electronic device may acquire the finger position recognition model from the memory of the electronic device. In other embodiments, the finger position recognition model may also be stored in a cloud server, so that after the reflected ultrasonic signal is acquired through the microphone of the electronic device, the processor of the electronic device may acquire the finger position recognition model from the cloud server.

[0067] It can be understood that the ultrasonic signal is a sound signal of a certain frequency, so the ultrasonic signal has a certain period, which is the inverse of the frequency. The amplitude corresponding to each period of the reflected ultrasonic signal can be extracted through the finger position recognition model. Each period and the amplitude corresponding to the period can be used as the characteristic parameters corresponding to the ultrasonic signal.

[0068] It should be noted that, when the designated parameter includes the amplitude, the reference parameter may also include the amplitude, so that the characteristic parameter can be compared with a plurality of preset reference parameters later.

[0069] Optionally, for some other implementations, the specified parameters may also include spectrum distribution. Thus, step S230 may also include step S233 and step S235.

[0070] Step S233: performing noise reduction processing on the reflected ultrasonic signal through the finger position recognition model.

[0071] Step S234: performing a time-frequency conversion operation on the ultrasonic signal after the noise reduction processing to obtain a frequency domain signal corresponding to the ultrasonic signal.

[0072] Step S235: performing spectrum analysis on the frequency domain signal to obtain spectrum features as characteristic parameters corresponding to the ultrasonic signal.

[0073] Taking the case where the designated parameters include frequency spectrum features as an example, the reflected ultrasonic signal can be subjected to a time-frequency conversion operation through a finger position recognition model.

[0074] Optionally, in order to improve the accuracy of the frequency domain signal obtained subsequently, the reflected ultrasonic signal may first be subjected to noise reduction processing by means of a finger position recognition model. Exemplarily, the noise reduction processing may include methods such as filtering, deep learning noise reduction or wavelet transform.

[0075] Noise reduction processing can suppress the noise components in the ultrasonic signal to a certain extent, so as to improve the accuracy and effectiveness of subsequent time-frequency conversion operations.

[0076] Further, the ultrasonic signal after noise reduction processing may be subjected to a time-frequency conversion operation to obtain a frequency domain signal corresponding to the ultrasonic signal. Exemplarily, the ultrasonic signal after noise reduction processing may be subjected to a Fourier transform (FT) to obtain a frequency domain signal corresponding to the ultrasonic signal.

[0077] Then, the frequency domain signal may be subjected to spectrum analysis to obtain spectrum features as characteristic parameters corresponding to the ultrasonic signal. The spectrum analysis of the frequency domain signal may also be implemented through a finger position recognition model.

[0078] The spectral characteristics refer to the properties of the ultrasonic signal in the frequency domain, which can be used to describe the energy distribution and phase information of the ultrasonic signal at different frequencies.

[0079] It can be understood that in order to achieve the situation where the specified parameters are at least one of amplitude and spectral distribution, the characteristic parameters corresponding to the ultrasonic signal are obtained through the finger position recognition model, multiple fully connected layers can be set for the specified parameters, and different characteristic parameters can be output through different activation functions.

[0080] Optionally, when the specified parameters are amplitude and frequency spectrum, the method of obtaining characteristic parameters corresponding to the ultrasonic signal through the finger position recognition model can refer to the introduction of the aforementioned steps, which will not be repeated here.

[0081] Step S240: Compare the characteristic parameter with a plurality of preset reference parameters, and determine a reference parameter matching the characteristic parameter as a target reference parameter, wherein each of the reference parameters corresponds to a reference position area.

[0082] Step S250: Determine a reference position area corresponding to the target reference parameter as the position area where the user's finger is currently located on the display screen of the electronic device.

[0083] Therefore, after the characteristic parameters are obtained, the characteristic parameters can be compared with a plurality of preset reference parameters, and then the reference parameters matching the characteristic parameters can be determined as the target reference parameters.

[0084] For some embodiments, when the characteristic parameter includes an amplitude, the characteristic parameter can be compared with the amplitude in each preset reference parameter to obtain the amplitude of the reference parameter closest to the amplitude of the characteristic parameter, and the reference parameter is the target reference parameter. Exemplarily, the difference between the amplitude corresponding to each period in the characteristic parameter and the amplitude corresponding to each period in the reference parameter can be compared to obtain the difference corresponding to each period, and then the difference corresponding to each period is summed. The smaller the sum obtained, the higher the degree of matching between the reference parameter and the characteristic parameter, and the reference parameter with the highest degree of matching can be determined as the target reference parameter.

[0085] For some other embodiments, when the characteristic parameters include spectral characteristics, similar to the above-mentioned case where the characteristic parameters include amplitude, the characteristic parameters can also be compared with the spectral parameters in each preset reference parameter to obtain the spectral parameters of the reference parameter that are closest to the amplitude of the characteristic parameters, and the reference parameter is the target reference parameter.

[0086] As can be seen from the above introduction, each of the reference parameters corresponds to a reference position area, that is, the reference position area corresponding to the target reference parameter can be used as the position area where the user's finger is currently located on the display screen of the electronic device.

[0087] Step S260: When the position area is a preset area, control the vibration motor to operate.

[0088] Among them, step S260 has been introduced in detail in the above embodiment and will not be repeated here.

[0089] The vibration control method provided by the embodiment of the present application analyzes the designated parameters of the reflected ultrasonic signal through the finger position recognition model to obtain the characteristic parameters corresponding to the ultrasonic signal; compares the characteristic parameters with a plurality of preset reference parameters to determine the reference parameters matching the characteristic parameters as the target reference parameters, wherein each of the reference parameters corresponds to a reference position area; determines the reference position area corresponding to the target reference parameter as the position area where the user's finger is currently located on the display screen of the electronic device. Wherein, the designated parameters include at least one of amplitude and spectrum distribution, so that the target parameters are determined by comparing the designated parameters with the reference parameters, and then the position area is further determined, and the accuracy of the obtained position area is high. In addition, the noise component in the ultrasonic signal can be suppressed to a certain extent by noise reduction processing to improve the accuracy and effectiveness of subsequent time-frequency conversion operations, thereby further improving the accuracy of the position area. Thus, the vibration delay can be reduced while the position area can be determined with high accuracy.

[0090] See also Figure 7 , Figure 7 A method flow chart of a vibration control method provided by an embodiment of the present application is shown. The vibration control method can be applied to Figure 1 The electronic device in the vibration control scenario shown can specifically use a processor of the electronic device as an execution subject for executing the vibration control method. The vibration control method can include steps S310 to S360.

[0091] Step S310: Obtain the application type corresponding to the screen content displayed on the display screen of the electronic device.

[0092] Step S311: Whether the application type is a specified type.

[0093] Electronic devices can run different applications, such as game applications, video applications, music applications, or schedule reminder applications, that is, applications can be of different types, for example, the types may include games, videos, music, schedule reminders, etc. Users can interact with the display screen of the electronic device, and then the electronic device can generate vibrations to generate vibration feedback to the user's interaction in addition to responding to the user's interactive operations, providing the user with a better interactive experience. It is understandable that different types of applications have different delay requirements for generating vibration feedback. For example, game applications have higher delay requirements for generating vibration feedback; video applications and music applications have lower delay requirements for generating vibration feedback; and schedule reminder applications may not need to generate vibrations.

[0094] Therefore, in some implementations, the application type may be determined first; and then, based on the acquired application type, it may be determined whether an ultrasonic signal needs to be emitted through a speaker.

[0095] Specifically, the application type corresponding to the screen content displayed on the display screen of the electronic device may be first obtained. Exemplarily, the application type corresponding to the screen content currently displayed on the display screen may be analyzed from the operation log of the electronic device; in another exemplary embodiment, the screen content displayed on the display screen of the electronic device may be intercepted, for example, multiple frames of screen content may be intercepted, and then image analysis may be performed on the intercepted multiple frames of screen content to determine the corresponding application type.

[0096] Furthermore, after determining the application type, it is possible to determine whether the application type belongs to a specified type, and if the application type belongs to the specified type, jump to step S320 to emit an ultrasonic signal through the speaker of the electronic device; and if the application type does not belong to the specified type, jump to step S360.

[0097] The specified type includes applications whose requirements for a delay parameter are less than or equal to a specified threshold, and the delay parameter is the delay for generating vibration feedback. That is, when the application type belongs to the specified type, it can be characterized that the application type has a small requirement for the delay parameter.

[0098] Step S320: When the application type belongs to a specified type, an ultrasonic signal is emitted through a speaker of the electronic device, wherein the specified type includes a type of application whose requirement for a delay parameter is less than or equal to a specified threshold.

[0099] The designated type may be a pre-set type, and may be determined based on the magnitude relationship between the requirements of each application for the delay parameter and the designated threshold. Specifically, the type of application whose requirements for the delay parameter are less than or equal to the designated threshold may be the designated type; and the type of application whose requirements for the delay parameter are greater than the designated threshold may be the non-designated type.

[0100] Therefore, if the application type acquired through the aforementioned steps belongs to a specified type, an ultrasonic signal can be emitted through the speaker of the electronic device and subsequent steps can be performed.

[0101] The application type belonging to the specified type may be any type of the specified type.

[0102] Optionally, the display screen of the electronic device can also be split-screen, so that the display screen of the electronic device can display multiple types of applications at the same time. For example, the left part of the display screen of the electronic device displays an audio application, and the right part of the display screen displays a video application.

[0103] Therefore, the application type obtained through the above steps may also include multiple types, so that the application type belongs to the specified type or there is at least one type in the application type that is a type in the specified type.

[0104] Step S330: When the user's finger interacts with the display screen, the reflected ultrasonic signal is acquired through the microphone of the electronic device.

[0105] Step S340: Based on the pre-trained finger position recognition model, the position area where the user's finger is currently interacting with the display screen of the electronic device is identified through the reflected ultrasonic signal.

[0106] Among them, step S330 and step S340 have been introduced in detail in the above embodiments and will not be repeated here.

[0107] Step S350: When the position area is a preset area, control the vibration motor to operate.

[0108] For some embodiments, the preset area may include the location area where the operation icon for releasing the specified skill is located in the game application. Exemplarily, for a game application of the Multiplayer Online Battle Arena (MOBA) type, the preset area may be the location area where the operation icon for releasing the ultimate skill is located. Thus, the user's finger can slide to the preset area, and then press the operation icon for releasing the specified skill. The electronic device identifies the location area where the user's finger is located, and when the user's finger slides to the preset area, the vibration motor can be controlled to work, thereby achieving vibration control at almost the same time as the user presses the operation icon for releasing the specified skill, with low vibration delay.

[0109] It is understandable that since controlling the operation of the vibration motor consumes a lot of electricity, for some embodiments, before controlling the operation of the vibration motor, the remaining power of the electronic device can also be detected, so that the vibration motor is controlled to operate only when the remaining power is high.

[0110] Specifically, step S350 may include step S351 and step S352.

[0111] Step S351: when the location area is a preset area, obtaining the remaining power of the electronic device.

[0112] Step S352: When the remaining power is greater than or equal to the power threshold, control the vibration motor to operate.

[0113] When the location area is a preset area, the remaining power of the electronic device can be obtained. For example, the current remaining power can be obtained through the system status log of the electronic device; for another example, the display screen of the electronic device generally displays a power indicator, so the display screen can be screenshotted, and then the screenshot can be analyzed to determine the remaining power of the electronic device.

[0114] Furthermore, it is possible to judge whether the remaining power is greater than or equal to a power threshold. When the remaining power is greater than or equal to the power threshold, the vibration motor is controlled to operate; and when the remaining power is less than the power threshold, the vibration motor can be controlled not to operate.

[0115] The power threshold may be pre-set. For example, a critical power level at which the power level after the vibration motor is controlled to work will not affect the normal operation of the electronic device may be obtained in advance as the power threshold. Thus, when the remaining power level is greater than or equal to the power threshold, even if the vibration motor is subsequently controlled to work, the normal operation of the electronic device will not be affected after the vibration motor is operated, thereby improving the stability of the operation of the electronic device.

[0116] Optionally, from the foregoing introduction, it can be known that controlling the vibration motor to work is to provide the user with better vibration feedback to enhance the user experience. However, when using an electronic device, the user may place the electronic device on an object instead of holding the electronic device, such as placing it on a desktop. Therefore, if the vibration motor is controlled to work at this time, it will not improve the user experience and will also cause a waste of electricity. Therefore, for some embodiments, before controlling the vibration motor of the electronic device to work, the holding state of the electronic device can also be detected, so that the vibration motor is controlled to work only when the holding state is holding.

[0117] Specifically, step S350 may also include step S353 and step S354.

[0118] Step S353: when the location area is a preset area, obtaining the holding state of the electronic device.

[0119] Step S354: When the holding state is holding, controlling the vibration motor to operate.

[0120] When the position area is a preset area, the holding state of the electronic device can be obtained. For example, the posture of the electronic device can be detected by a gyroscope or an inertial measurement unit configured in the electronic device, so as to determine the holding state of the electronic device according to the posture. The holding state is used to indicate whether the electronic device is held by the user, and the holding state includes being held or not held.

[0121] Then, when the holding state is holding, the vibration motor is controlled to work; when the holding state is not holding, the vibration motor can be controlled not to work.

[0122] Thus, it can be determined that the electronic device is held by the user when the vibration motor is working, thereby ensuring that the vibration of the vibration motor can form vibration feedback to the user.

[0123] Step S360: Controlling the operation of the vibration motor of the electronic device by detecting the interaction interruption.

[0124] It is understandable that electronic devices generate and send ultrasonic signals, which consumes a lot of electricity. Therefore, if the application type does not have a low requirement for latency, the vibration motor can be controlled without sending and obtaining reflected ultrasonic signals.

[0125] Specifically, when the application type does not belong to the specified type, it can be characterized that the application type has a greater demand for delay, so that it is not necessary to control the operation of the vibration motor by sending and obtaining the reflected ultrasonic signal. In some embodiments, the operation of the vibration motor of the electronic device can be controlled by detecting the interaction interruption.

[0126] If an interaction interruption is detected, it can represent that a user input to the electronic device is detected, so that the vibration motor of the electronic device can be controlled to work when the interaction interruption is detected. Exemplarily, the interaction interruption can be captured by the processor from the operation status log of the electronic device.

[0127] It is understandable that the method of controlling the operation of the vibration motor of the electronic device by detecting the interaction interruption has a higher delay than the method of controlling the operation of the vibration motor by sending and obtaining the reflected ultrasonic signal, but can bring lower energy consumption.

[0128] In the implementation provided by the present application, the application type corresponding to the screen content displayed on the display screen of the electronic device can be obtained; when the application type belongs to the specified type, an ultrasonic signal is emitted through the speaker of the electronic device, wherein the specified type includes the type of application whose requirement for delay parameter is less than or equal to the specified threshold. In the case where the application type does not belong to the specified type, it is characterized that the application type has a large requirement for delay, so that the vibration motor of the electronic device is controlled to work by detecting the interaction interruption, which will not exceed the requirement for delay of the application type on the one hand, and save the energy consumption of the electronic device on the other hand. Furthermore, the vibration motor can be controlled to work when the remaining power is greater than or equal to the power threshold, and even if the vibration motor is subsequently controlled to work, it will not affect the normal operation of the electronic device after the vibration motor works, thereby improving the stability of the operation of the electronic device. The vibration motor can also be controlled to work when the holding state is holding. Thus, it can be determined that the electronic device is held by the user when the vibration motor is working, thereby ensuring that the vibration of the vibration motor can form vibration feedback to the user.

[0129] See also Figure 8 , Figure 8 A structural block diagram of a vibration control device provided in an embodiment of the present application is shown. The vibration control device 800 includes: a transmitting unit 810, an acquiring unit 820, a recognition unit 830 and a control unit 840.

[0130] The transmitting unit 810 is used to transmit an ultrasonic signal through a speaker of the electronic device.

[0131] Optionally, the transmitting unit 810 can also be used to obtain the application type corresponding to the screen content displayed on the display screen of the electronic device; when the application type belongs to a specified type, an ultrasonic signal is emitted through the speaker of the electronic device, wherein the specified type includes the type of application whose requirement for a delay parameter is less than or equal to a specified threshold.

[0132] The acquisition unit 820 is used to acquire the reflected ultrasonic signal through the microphone of the electronic device when the user's finger interacts with the display screen.

[0133] The identification unit 830 is used to identify the location area where the user's finger is currently interacting with the display screen of the electronic device based on the pre-trained finger position identification model through the reflected ultrasonic signal.

[0134] Optionally, the recognition unit 830 can also be used to analyze the specified parameters of the reflected ultrasonic signal through the finger position recognition model to obtain characteristic parameters corresponding to the ultrasonic signal; compare the characteristic parameters with a plurality of preset reference parameters to determine reference parameters matching the characteristic parameters as target reference parameters, wherein each reference parameter corresponds to a reference position area; and determine the reference position area corresponding to the target reference parameter as the position area where the user's finger is currently located on the display screen of the electronic device. The specified parameters include at least one of amplitude and spectrum distribution.

[0135] Optionally, the recognition unit 830 can also be used to extract the amplitude corresponding to each cycle of the reflected ultrasonic signal through the finger position recognition model; and use each cycle and the amplitude corresponding to the cycle as characteristic parameters corresponding to the ultrasonic signal.

[0136] Optionally, the recognition unit 830 can also be used to perform noise reduction processing on the reflected ultrasonic signal through the finger position recognition model; perform time-frequency conversion on the ultrasonic signal after noise reduction processing to obtain a frequency domain signal corresponding to the ultrasonic signal; perform spectrum analysis on the frequency domain signal to obtain spectrum characteristics as characteristic parameters corresponding to the ultrasonic signal.

[0137] The control unit 840 is used to control the vibration motor to work when the position area is a preset area, wherein the preset area includes the position area where the operation icon for releasing a specified skill in the game application is located.

[0138] Optionally, the control unit 840 can also be used to obtain the remaining power of the electronic device when the location area is a preset area; and control the vibration motor to operate when the remaining power is greater than or equal to a power threshold.

[0139] Optionally, the control unit 840 may also be configured to obtain a holding state of the electronic device when the position area is a preset area; and control the vibration motor to operate when the holding state is holding.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0141] In several embodiments provided in the present application, the coupling between the units may be electrical, mechanical or other forms of coupling. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0142] See also Fig. 9 , Fig. 9 The structural block diagram of an electronic device provided in an embodiment of the present application is shown. The electronic device 110 may be a smart phone or a tablet computer, etc. The electronic device 110 in the present application may include one or more of the following components: a processor 111, a memory 112, a speaker 113, and a microphone 114, wherein the processor 111 is connected to the memory 112, the speaker 113, and the microphone 114, respectively, and the processor 111 is used to execute the methods described in the above embodiments.

[0143] The processor 111 may include one or more processing cores. The processor 111 uses various interfaces and lines to connect various parts of the entire electronic device 110, and executes various functions and processes data of the electronic device 110 by running or executing instructions, programs, code sets or instruction sets stored in the memory 112, and calling data stored in the memory 112. Optionally, the processor 111 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 111 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and computer programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 111, but may be implemented separately through a communication chip. Specifically, the method described in the above embodiments may be executed by one or more processors 111 .

[0144] For some embodiments, the memory 112 may include a random access memory (RAM) or a read-only memory (ROM). The memory 112 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 112 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the electronic device 110 during use, etc.

[0145] The speaker 113 may be a micro speaker. The microphone 114 may include one or more microphones, wherein a plurality of microphones may form a microphone array.

[0146] Optional, please continue to Fig. 9 , Fig. 9The ultrasonic wave generator 110 may further include an audio power amplifier 115, which may be connected to the processor 111 and the speaker 113 respectively. The processor 111 sends the audio signal corresponding to the ultrasonic wave signal to the audio power amplifier 115, and the audio power amplifier 115 amplifies the audio signal and outputs it through the speaker 113.

[0147] Optional, please continue to Fig. 9 , Fig. 9 The apparatus may further include a codec 116, which is connected to the processor 111 and the microphone 114 respectively. Thus, the reflected ultrasonic signal obtained by the microphone 114 can be input into the codec 116, and the codec 116 encodes the reflected ultrasonic signal to obtain a coded signal corresponding to the reflected ultrasonic signal, and then transmits the coded signal to the processor 111.

[0148] See also Fig.10 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 1000 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0149] The computer readable storage medium 1000 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 1000 includes a non-transitory computer-readable storage medium. The computer readable storage medium 1000 has storage space for program code 1010 that executes any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 1010 can be compressed, for example, in an appropriate form.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A vibration control method, It is characterized in that A processor applied to an electronic device, the electronic device further comprising a vibration motor, the processor being connected to the vibration motor, the method comprising: emitting an ultrasonic signal through a speaker of the electronic device; When the user's finger interacts with the display screen, the reflected ultrasonic signal is acquired through the microphone of the electronic device; Based on a pre-trained finger position recognition model, the position area where the user's finger currently interacts with the display screen of the electronic device is identified by the reflected ultrasonic signal; When the position area is a preset area, the vibration motor is controlled to operate.

2. The method according to claim 1, It is characterized in that The method of identifying the position area where the user's finger currently interacts with the display screen of the electronic device based on the pre-trained finger position recognition model by reflecting the ultrasonic signal includes: By using the finger position recognition model, the specified parameters of the reflected ultrasonic signal are analyzed to obtain characteristic parameters corresponding to the ultrasonic signal; Comparing the characteristic parameter with a plurality of preset reference parameters, determining a reference parameter matching the characteristic parameter as a target reference parameter, wherein each of the reference parameters corresponds to a reference position area; A reference position area corresponding to the target reference parameter is determined as the position area where the user's finger is currently located on the display screen of the electronic device.

3. The method according to claim 2, It is characterized in that The specified parameter includes at least one of amplitude and frequency spectrum distribution.

4. The method according to claim 3, It is characterized in that The designated parameter includes amplitude, and the designated parameter of the reflected ultrasonic signal is analyzed by the finger position recognition model to obtain characteristic parameters corresponding to the ultrasonic signal, including: Extracting the amplitude corresponding to each cycle of the reflected ultrasonic signal through the finger position recognition model; Each period and the amplitude corresponding to the period are used as characteristic parameters corresponding to the ultrasonic signal.

5. The method according to claim 3, It is characterized in that The designated parameters include frequency spectrum distribution, and the designated parameters of the reflected ultrasonic signal are analyzed by the finger position recognition model to obtain characteristic parameters corresponding to the ultrasonic signal, including: Performing noise reduction processing on the reflected ultrasonic signal through the finger position recognition model; Performing a time-frequency conversion operation on the ultrasonic signal after noise reduction processing to obtain a frequency domain signal corresponding to the ultrasonic signal; Performing spectrum analysis on the frequency domain signal to obtain spectrum features as characteristic parameters corresponding to the ultrasonic signal.

6. The method according to claim 1, It is characterized in that The transmitting of the ultrasonic signal by the speaker of the electronic device comprises: Obtaining the application type corresponding to the screen content displayed on the display screen of the electronic device; In a case where the application type belongs to a specified type, an ultrasonic signal is emitted through a speaker of the electronic device, wherein the specified type includes a type of application whose requirement for a delay parameter is less than or equal to a specified threshold.

7. The method according to claim 1, It is characterized in that When the position area is a preset area, controlling the vibration motor to operate includes: When the location area is a preset area, obtaining the remaining power of the electronic device; When the remaining power is greater than or equal to a power threshold, the vibration motor is controlled to operate.

8. The method according to claim 1, It is characterized in that When the position area is a preset area, controlling the vibration motor to operate includes: When the location area is a preset area, obtaining a holding state of the electronic device; When the held state is holding, the vibration motor is controlled to operate.

9. The method according to claim 1, It is characterized in that The preset area includes a location area where an operation icon for releasing a specified skill in a game application is located.

10. A vibration control device, It is characterized in that A processor applied to an electronic device, wherein the electronic device further comprises a vibration motor, the processor is connected to the vibration motor, and the device comprises: A transmitting unit, used for transmitting an ultrasonic signal through a speaker of the electronic device; an acquisition unit, configured to acquire the reflected ultrasonic signal through a microphone of the electronic device when a user's finger interacts with the display screen; an identification unit, configured to identify a location area where the user's finger is currently interacting with the display screen of the electronic device through the reflected ultrasonic signal based on a pre-trained finger location identification model; The control unit is used to control the vibration motor to operate when the position area is a preset area.

11. An electronic device, It is characterized in that The electronic device comprises a processor, a vibration motor, a speaker and a microphone, wherein the processor is connected to the vibration motor, the microphone and the speaker respectively; The processor is configured to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that The readable storage medium stores program code, and the program code can be called by a processor to execute the method according to any one of claims 1 to 9.

Images