Method, device and storage medium for identifying an obstacle action

Abstract

This disclosure relates to a method, apparatus, and storage medium for recognizing obstacle movements. For each received frame of ultrasonic echo signal, feature information is extracted from that frame, including phase information. The ultrasonic echo signal is the reflected signal of an emitted ultrasonic signal. Based on the phase information, it is determined whether a target movement event has occurred. The target movement event includes a first event where an obstacle approaches a terminal or a second event where the obstacle moves away from the terminal. If the target movement event is determined to have occurred, a preset operation is executed, with different preset operations corresponding to different target movement events.

Description

Technical Field

[0001] This disclosure relates to the field of obstacle recognition, and more particularly to a method, apparatus and storage medium for recognizing obstacle movements. Background Technology

[0002] In related technologies, the anti-accidental touch function of the terminal is mostly realized through infrared distance sensors. The infrared distance sensor emits infrared signals and detects the changes in the reflected infrared signals returned to the sensor to calculate the distance between the terminal and the obstacle, thereby determining whether the terminal is close to the obstacle, and then realizing the anti-accidental touch function of the terminal based on the judgment result. In another solution, an ultrasonic sine wave signal is emitted from an ultrasonic transmitter, and an ultrasonic receiver receives the ultrasonic sine wave signal through the air. Then, it detects when the frequency of the transmitted ultrasonic sine wave signal and the frequency of the received ultrasonic sine wave signal meet a predetermined difference standard, and then determines whether to disable or enable touch or non-touch input on the terminal. Summary of the Invention

[0003] To overcome the problems existing in the related technologies, this disclosure provides a method, apparatus and storage medium for recognizing obstacle movements.

[0004] According to a first aspect of the present disclosure, a method for identifying obstacle actions is provided, comprising: extracting features from each received frame of ultrasonic echo signal to obtain feature information, the feature information including phase information; the ultrasonic echo signal being a reflected signal of an emitted ultrasonic signal; determining whether a target action event has occurred based on the phase information, the target action event including a first event of an obstacle approaching a terminal or a second event of an obstacle moving away from the terminal; and, if the target action event is determined to have occurred, performing a preset operation, with different preset operations corresponding to different target action events.

[0005] Optionally, the step of extracting feature information from the ultrasonic echo signal frame includes: performing a frequency domain transformation on the ultrasonic echo signal frame to obtain a frequency domain signal; and extracting the phase information corresponding to the ultrasonic echo signal frame based on the real and imaginary parts of the frequency domain signal.

[0006] Optionally, determining whether a target action event has occurred based on the phase information includes: acquiring the phase information corresponding to multiple frames of ultrasonic echo signals received within a first preset time period; calculating the phase difference between a first phase and a second phase, wherein the first phase includes the phase information corresponding to the last frame of ultrasonic echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasonic echo signal within the first preset time period; determining that the first event has occurred if the phase difference is greater than or equal to a first preset threshold; and determining that the second event has occurred if the phase difference is less than or equal to a second preset threshold.

[0007] Optionally, determining whether a target action event has occurred based on the phase information includes: inputting the phase information corresponding to a preset number of ultrasonic echo signals into a pre-trained neural network model to obtain the model output; and determining whether the target action event has occurred based on the model output.

[0008] Optionally, the feature information further includes at least one of volume information and frequency shift information. The frequency shift information includes a first frequency shift parameter and a second frequency shift parameter. The feature extraction of the ultrasonic echo signal frame to obtain the feature information includes: obtaining the first transmitted wave energy corresponding to the highest frequency of the transmitted wave and the first energy of the first other signal based on the spectrum of the ultrasonic echo signal frame, wherein the first other signal includes a preset number of signals with frequencies higher than the highest frequency of the transmitted wave; calculating the first frequency shift parameter based on the first transmitted wave energy and the first energy; obtaining the second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and the second energy of the second other signal based on the spectrum of the ultrasonic echo signal frame, wherein the second other signal includes a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave; and calculating the second frequency shift parameter based on the second transmitted wave energy and the second energy.

[0009] Optionally, determining whether a target action event has occurred based on the phase information includes: determining whether the target action event has occurred based on the phase information and preset information, wherein the preset information includes at least one of the volume information and the frequency shift information.

[0010] Optionally, determining whether the target action event has occurred based on the phase information and preset information includes: determining that the first event has occurred if the phase information satisfies a first preset condition and the preset information satisfies a second preset condition; and determining that the second event has occurred if the phase information satisfies a third preset condition and the preset information satisfies a fourth preset condition.

[0011] Optionally, the method includes: acquiring phase information corresponding to multiple frames of ultrasound echo signals received within a first preset time period; calculating the phase difference between a first phase and a second phase, wherein the first phase includes the phase information corresponding to the last frame of ultrasound echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasound echo signal within the first preset time period; determining that the phase information satisfies the first preset condition when the phase difference is greater than or equal to a first preset threshold, and determining that the phase information satisfies the third preset condition when the phase difference is less than or equal to a second preset threshold.

[0012] Optionally, the second preset condition includes a first volume condition and / or a first frequency shift condition; the fourth preset condition includes a second volume condition and / or a second frequency shift condition; the method includes: if the volume information satisfies the first volume condition and / or the frequency shift information satisfies the first frequency shift condition, determining that the preset information satisfies the second preset condition; if the volume information satisfies the second volume condition and / or the frequency shift information satisfies the second frequency shift condition, determining that the preset information satisfies the fourth preset condition;

[0013] Determining whether the volume information meets the first volume condition or the second volume condition includes:

[0014] If the volume information satisfies a preset volume sub-condition, the volume change slope of the ultrasonic echo signal within a second preset time period is calculated based on the volume information. If the volume change slope is positive, the volume information is determined to satisfy the first volume condition. If the volume change slope is negative, the volume information is determined to satisfy the second volume condition.

[0015] or,

[0016] If the volume information meets a preset volume sub-condition, determine the trend of the volume peak value on the volume curve within the second preset time period. If the trend of ...

[0017] or,

[0018] If the volume information satisfies a preset volume sub-condition, a linear function is performed on the volume peak to obtain a fitting function; if the slope of the fitting function is positive, the volume information is determined to satisfy the first volume condition; if the slope of the fitting function is negative, the volume information is determined to satisfy the second volume condition.

[0019] The preset volume sub-conditions include:

[0020] The volume variance is greater than or equal to a third preset threshold, where the volume variance is the variance corresponding to the volume information of the ultrasonic echo signal received within a second preset time period; or, the sum of the peak-to-trough differences is greater than or equal to a fourth preset threshold, where the sum of the peak-to-trough differences is the sum of the peak-to-trough differences of the volume curve of the ultrasonic echo signal received within the second preset time period; or, the difference between the maximum volume and the minimum volume is greater than or equal to a fifth preset threshold, where the difference between the maximum volume and the minimum volume is the difference between the maximum volume and the minimum volume of the ultrasonic echo signal received within the second preset time period.

[0021] Determining whether the frequency shift information satisfies the first frequency shift condition or the second frequency shift condition includes:

[0022] If the first frequency shift parameter is greater than or equal to a sixth preset threshold, and the first frequency shift parameter is greater than or equal to K times the second frequency shift parameter, then the frequency shift information is determined to satisfy the first frequency shift condition, where K is a preset number.

[0023] If the second frequency shift parameter is greater than or equal to a seventh preset threshold, and the second frequency shift parameter is greater than or equal to the first frequency shift parameter multiplied by K, then the frequency shift information is determined to satisfy the second frequency shift condition.

[0024] Optionally, determining whether the target action event has occurred based on the phase information and preset information includes: inputting the phase information corresponding to the ultrasonic echo signal of a preset number of frames and the preset information into a pre-trained neural network model to obtain the model output; and determining whether the target action event has occurred based on the model output.

[0025] Optionally, the step of performing a preset operation when the target action event is determined to have occurred includes: when the first event is determined to have occurred, controlling the terminal to disable touch operation; and when the second event is determined to have occurred, controlling the terminal to enable touch operation.

[0026] Optionally, the step of performing a preset operation when the target action event is determined to have occurred includes: controlling the terminal to enable a preset function when the first event is determined to have occurred; and controlling the terminal to disable the preset function when the second event is determined to have occurred.

[0027] According to a second aspect of the present disclosure, an apparatus for recognizing obstacle movement is provided, comprising:

[0028] The feature extraction module is configured to extract feature information from each received frame of ultrasonic echo signal to obtain feature information, including phase information; the ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal.

[0029] The first determining module is configured to determine whether a target action event has occurred based on the phase information. The target action event includes a first event in which an obstacle approaches the terminal or a second event in which the obstacle moves away from the terminal.

[0030] The action execution module is configured to perform a preset operation when the target action event is determined to have occurred, with different preset operations corresponding to different target action events.

[0031] Optionally, the feature extraction module is configured to perform frequency domain transformation on the frame of ultrasonic echo signal to obtain a frequency domain signal; and extract the phase information corresponding to the frame of ultrasonic echo signal based on the real and imaginary parts of the frequency domain signal.

[0032] Optionally, the first determining module is configured to acquire the phase information corresponding to multiple frames of ultrasonic echo signals received within a first preset time period; calculate the phase difference between a first phase and a second phase, wherein the first phase includes the phase information corresponding to the last frame of ultrasonic echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasonic echo signal within the first preset time period; determine that the first event has occurred if the phase difference is greater than or equal to a first preset threshold; and determine that the second event has occurred if the phase difference is less than or equal to a second preset threshold.

[0033] Optionally, the first determining module is configured to input the phase information corresponding to the ultrasonic echo signals of a preset number of frames into a pre-trained neural network model to obtain the model output; and determine whether the target action event has occurred based on the model output.

[0034] Optionally, the feature information further includes at least one of volume information and frequency shift information. The frequency shift information includes a first frequency shift parameter and a second frequency shift parameter. The feature extraction module is configured to obtain, based on the spectrum of the ultrasonic echo signal frame, a first transmitted wave energy corresponding to the highest frequency of the transmitted wave and a first energy of a first other signal, wherein the first other signal includes a preset number of signals with frequencies higher than the highest frequency of the transmitted wave; calculate the first frequency shift parameter based on the first transmitted wave energy and the first energy; obtain, based on the spectrum of the ultrasonic echo signal frame, a second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and a second energy of a second other signal, wherein the second other signal includes a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave; and calculate the second frequency shift parameter based on the second transmitted wave energy and the second energy.

[0035] Optionally, the first determining module is configured to determine whether the target action event has occurred based on the phase information and preset information, wherein the preset information includes at least one of the volume information and the frequency shift information.

[0036] Optionally, the first determining module is configured to determine that the first event has occurred when the phase information satisfies a first preset condition and the preset information satisfies a second preset condition; and to determine that the second event has occurred when the phase information satisfies a third preset condition and the preset information satisfies a fourth preset condition.

[0037] Optionally, the first determining module is configured to acquire the phase information corresponding to multiple frames of ultrasonic echo signals received within a first preset time period; calculate the phase difference between a first phase and a second phase, wherein the first phase includes the phase information corresponding to the last frame of ultrasonic echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasonic echo signal within the first preset time period; determine that the phase information satisfies the first preset condition if the phase difference is greater than or equal to a first preset threshold, and determine that the phase information satisfies the third preset condition if the phase difference is less than or equal to a second preset threshold.

[0038] Optionally, the second preset condition includes a first volume condition and / or a first frequency shift condition; the fourth preset condition includes a second volume condition and / or a second frequency shift condition; the first determining module is configured to determine that the preset information satisfies the second preset condition when the volume information satisfies the first volume condition and / or the frequency shift information satisfies the first frequency shift condition; and to determine that the preset information satisfies the fourth preset condition when the volume information satisfies the second volume condition and / or the frequency shift information satisfies the second frequency shift condition.

[0039] Specifically, when the volume information meets a preset volume sub-condition, the volume change slope of the ultrasonic echo signal within a second preset time period is calculated based on the volume information. If the volume change slope is positive, the volume information is determined to meet the first volume condition. If the volume change slope is negative, the volume information is determined to meet the second volume condition.

[0040] or,

[0041] If the volume information meets a preset volume sub-condition, determine the trend of the volume peak value on the volume curve within the second preset time period. If the trend of ...

[0042] or,

[0043] If the volume information satisfies a preset volume sub-condition, a linear function is performed on the volume peak to obtain a fitting function; if the slope of the fitting function is positive, the volume information is determined to satisfy the first volume condition; if the slope of the fitting function is negative, the volume information is determined to satisfy the second volume condition.

[0044] The preset volume sub-conditions include:

[0045] The volume variance is greater than or equal to a third preset threshold, where the volume variance is the variance corresponding to the volume information of the ultrasonic echo signal received within a second preset time period; or, the sum of the peak-to-trough differences is greater than or equal to a fourth preset threshold, where the sum of the peak-to-trough differences is the sum of the peak-to-trough differences of the volume curve of the ultrasonic echo signal received within the second preset time period; or, the difference between the maximum volume and the minimum volume is greater than or equal to a fifth preset threshold, where the difference between the maximum volume and the minimum volume is the difference between the maximum volume and the minimum volume of the ultrasonic echo signal received within the second preset time period.

[0046] If the first frequency shift parameter is greater than or equal to a sixth preset threshold, and the first frequency shift parameter is greater than or equal to K times the second frequency shift parameter, then the frequency shift information is determined to satisfy the first frequency shift condition, where K is a preset number.

[0047] If the second frequency shift parameter is greater than or equal to a seventh preset threshold, and the second frequency shift parameter is greater than or equal to the first frequency shift parameter multiplied by K, then the frequency shift information is determined to satisfy the second frequency shift condition.

[0048] Optionally, the first determining module is configured to: input the phase information corresponding to the ultrasonic echo signal of a preset number of frames and the preset information into a pre-trained neural network model to obtain the model output; and determine whether the target action event has occurred based on the model output.

[0049] Optionally, the action execution module is configured to: control the terminal to disable touch operation when the first event is determined to occur; and control the terminal to enable touch operation when the second event is determined to occur.

[0050] Optionally, the action execution module is configured to: control the terminal to enable a preset function when the first event is determined to occur; and control the terminal to disable the preset function when the second event is determined to occur.

[0051] According to a third aspect of the present disclosure, an apparatus for recognizing obstacle movement is provided, comprising:

[0052] processor;

[0053] Memory used to store processor-executable instructions;

[0054] The processor is configured to: extract feature information from each received frame of ultrasonic echo signal to obtain feature information, the feature information including phase information; the ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal;

[0055] Based on the phase information, it is determined whether a target action event has occurred, the target action event including a first event of an obstacle approaching the terminal or a second event of the obstacle moving away from the terminal;

[0056] When the target action event is determined to have occurred, a preset operation is executed, with different preset operations corresponding to different target action events.

[0057] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the method for recognizing obstacle actions provided in the first aspect of the present disclosure.

[0058] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: feature extraction is performed on each frame of received ultrasonic echo signal to obtain phase information, thereby determining whether a first event of an obstacle approaching the terminal or a second event of an obstacle moving away from the terminal has occurred based on the phase information. The phase information describes the phase change of the ultrasonic echo signal, and the phase change is only related to the distance change, and is not related to the speed or the frequency response of the terminal itself. Moreover, the phase feature is highly sensitive, that is, a small distance change can correspond to a large change in the phase feature. Therefore, based on the phase information of the ultrasonic echo signal, the action of an obstacle approaching / moving away from the terminal can be identified more accurately, and the robustness of the identified action is better. In turn, the terminal can be controlled to perform corresponding actions more accurately and stably according to the identification results, thereby realizing the terminal's anti-accidental touch function.

[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0061] Figure 1 This is a flowchart illustrating a method for recognizing obstacle actions according to an exemplary embodiment;

[0062] Figure 2 This is a schematic diagram of an architecture for recognizing obstacle actions according to an exemplary embodiment;

[0063] Figure 3 This is a phase change curve illustrating a process of a terminal approaching and moving away from an obstacle, according to an exemplary embodiment.

[0064] Figure 4 This is a flowchart illustrating a method for recognizing obstacle actions according to an exemplary embodiment;

[0065] Figure 5 This is a spectrum diagram of an ultrasonic echo signal according to an exemplary embodiment;

[0066] Figure 6 This is a flowchart illustrating a method for recognizing obstacle actions according to an exemplary embodiment;

[0067] Figure 7 This is a volume characteristic curve of an ultrasonic echo signal according to an exemplary embodiment;

[0068] Figure 8 This is a block diagram illustrating a device for recognizing obstacle movements according to an exemplary embodiment;

[0069] Figure 9 This is a block diagram illustrating a device for recognizing obstacle movements according to an exemplary embodiment. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0071] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0072] This disclosure can perform motion recognition of obstacles approaching / moving away from the terminal, so as to realize the terminal's anti-accidental touch function based on the recognition result. For example, when it is determined that the user's head is close to the mobile phone, the mobile phone can be controlled to disable touch input on the touch screen display during the call; or the terminal can be controlled to realize preset functions (such as turning on or off) based on the recognition result.

[0073] In related technologies, the anti-accidental touch function of the terminal is mostly achieved through infrared distance sensors. Infrared distance sensors emit infrared signals and detect changes in the reflected infrared signals returned to the sensor to calculate the distance between the terminal (such as mobile phones, iPads, etc.) and the obstacle, thereby determining whether the terminal is close to the obstacle. However, using infrared distance sensors has certain hardware costs and occupies a large space on the surface of the terminal. With the development of full-screen mobile phones, the screen ratio of mobile phones is getting higher and higher, and it is difficult to leave enough space at the top of the mobile phone to place an infrared distance sensor, so the distance sensing function cannot be completed in the traditional way.

[0074] In another related solution, the anti-accidental touch function of the terminal can be implemented by using a terminal equipped with an ultrasonic transmitter, an ultrasonic receiver, and a processing system. The ultrasonic transmitter emits an ultrasonic sine wave signal, and the ultrasonic receiver receives the ultrasonic sine wave signal through the air. Then, it detects when the frequency of the transmitted ultrasonic sine wave signal and the frequency of the received ultrasonic sine wave signal meet a predetermined difference standard, thereby determining whether to disable or enable touch or non-touch input on the terminal. However, in this solution, the ultrasonic transmitter usually emits a single-frequency ultrasonic signal, which has weak anti-interference capability. Furthermore, the frequency signal change is related to the moving speed of the obstacle. When the terminal moves away from / approaches the obstacle slowly, it cannot accurately identify the obstacle's movement based on the frequency shift signal. Therefore, detecting when the frequency of the transmitted ultrasonic sine wave signal and the frequency of the received ultrasonic sine wave signal meet the predetermined difference standard not only has poor anti-interference capability but also low recognition accuracy, making it difficult to meet the application requirements of the terminal.

[0075] To address the aforementioned problems, this disclosure provides a method, apparatus, and storage medium for identifying obstacle movements. It can determine whether a first event (obstacle approaching the terminal) or a second event (obstacle moving away from the terminal) has occurred based on the phase information of the received ultrasonic echo signal. The phase information describes the phase change of the ultrasonic echo signal, and this phase change is only related to distance changes, independent of speed magnitude and the terminal's own frequency response. Furthermore, the phase characteristic is highly sensitive; a small distance change can correspond to a large change in the phase characteristic. Therefore, based on the phase information of the ultrasonic echo signal, the movement of an obstacle approaching / moving away from the terminal can be identified more accurately, and the robustness of the identified movement is better. Consequently, the terminal can be controlled to perform corresponding actions more accurately and stably based on the identification results, thereby achieving the terminal's anti-accidental touch function.

[0076] In addition, considering that the process of identifying the approach / remote action through the feature information of the ultrasonic echo signal only requires the transmission and reception of the signal through the terminal's inherent sound-generating device (such as a speaker, microphone, etc.) and recording device (such as a microphone), the ultrasonic generator, feature extraction and action recognition are all implemented through software algorithms, without occupying additional device space, thus saving hardware costs.

[0077] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0078] Figure 1 This is a flowchart illustrating a method for recognizing obstacle actions according to an exemplary embodiment. This method can be applied to terminals (such as mobile phones, iPads, etc.). Figure 1 As shown, the method includes the following steps:

[0079] In step S101, for each received frame of ultrasonic echo signal, feature extraction is performed on the frame of ultrasonic echo signal to obtain feature information, which includes phase information.

[0080] The ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal.

[0081] For example, Figure 2 This is a schematic diagram of an architecture for recognizing obstacle actions according to an exemplary embodiment, such as... Figure 2 As shown, the architecture includes a transmitter and a receiver. The transmitter emits an ultrasonic signal of arbitrary period through an ultrasonic generator, and then emits the ultrasonic signal outward through a sound wave transmitter (such as a speaker, earpiece, etc.) on the terminal. The ultrasonic echo signal propagating from the air is received by a sound wave receiver (such as a microphone). Then, an analyzer can analyze each frame of the received ultrasonic echo signal and extract the corresponding feature information (such as phase information). After that, a recognizer can identify the terminal's movement of approaching / moving away from obstacles based on the feature information extracted by the analyzer.

[0082] Among them, the ultrasonic generator can be an ultrasonic signal exciter, which consists of two parts: an exciter and a filter. The ultrasonic signal exciter can generate multiple ultrasonic signals of different frequencies, which are converted into analog signals by a DAC digital-to-analog converter and then emitted to the outside world by a sound wave transmitter.

[0083] In this step, for each received ultrasonic echo signal, a frequency domain transformation can be performed on the ultrasonic echo signal to obtain a frequency domain signal; the phase information corresponding to the ultrasonic echo signal of that frame can be extracted based on the real and imaginary parts of the frequency domain signal.

[0084] In one possible implementation, the phase information can be obtained by calculating the arctangent of the real and imaginary parts of the frequency domain signal of the ultrasonic echo signal, as shown in the following formula:

[0085] ))

[0086] Where phase is the phase of the ultrasonic echo signal, atan2 is the arctangent function, F(w) is the Fourier transform of the ultrasonic echo signal, imag() indicates taking the imaginary part, and real() indicates taking the real part.

[0087] In step S102, it is determined whether a target action event has occurred based on the phase information. The target action event includes a first event in which an obstacle approaches the terminal or a second event in which the obstacle moves away from the terminal.

[0088] Considering practical application scenarios, when the distance between the terminal and an obstacle changes (i.e., there is relative motion), the phase of the ultrasonic echo signal received by the ultrasonic receiver will change. A sustained increase in the phase of the ultrasonic echo signal over a period of time indicates an approaching motion during that time, while a sustained decrease in the phase indicates a moving away motion during that time. Figure 3 The figure shown illustrates the phase change curve of a terminal approaching and moving away from an obstacle according to an exemplary embodiment. The phase characteristic is highly sensitive, meaning that a small distance change can correspond to a large change in the phase characteristic. For example, for a 20kHz ultrasonic signal, a distance change of 0.85cm can cause a phase change of πrad (half a cycle), as shown in the following formula:

[0089] Δφ=0.85cm / (1 / 20000Hz*34000cm / s)*2π=π

[0090] Therefore, based on the phase information of the ultrasonic echo signal, the movement of an obstacle approaching / moving away from the terminal can be identified more accurately, and the robustness of the identified movement is better. Thus, in one possible implementation of this step, the approaching / moving movement can be judged by calculating the cumulative phase change. For example, the phase information corresponding to multiple frames of ultrasonic echo signals received within a first preset time period can be obtained; the phase difference between a first phase and a second phase can be calculated, where the first phase includes the phase information corresponding to the last frame of ultrasonic echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasonic echo signal within the first preset time period; if the phase difference is greater than or equal to a first preset threshold, it indicates that the phase of the ultrasonic echo signal is continuously increasing within the first preset time period. Figure 3 As shown, this indicates that a proximity action occurred during this period, thus confirming the occurrence of the first event; if the phase difference is determined to be less than or equal to the second preset threshold, it indicates that the phase of the ultrasound echo signal continuously decreases during the first preset time period, such as... Figure 3 As shown, this indicates that a movement away occurred during this period, thus confirming the occurrence of the second event.

[0091] The first preset threshold and the second preset threshold can be any empirical value or can be set arbitrarily according to actual needs. This disclosure does not limit them. The first preset time period can be the time period during which a preset number of ultrasonic echo signals (such as 10 frames, 20 frames, etc.) are received. It should also be noted that when performing action recognition based on the phase information of the ultrasonic echo signals obtained in the next time period, it can partially overlap with the previous time period in time, thereby ensuring good continuity of action recognition. For example, assuming that the first preset time period is the time period during which 10 frames of ultrasonic echo signals are received, and the time overlap of different time periods is the reception time of 2 frames of signals, then the first action recognition can be based on the phase information of the ultrasonic echo signals received from the 1st frame to the 10th frame, the second action recognition can be based on the phase information of the ultrasonic echo signals received from the 3rd frame to the 12th frame, and so on, so that the approach / distance action can be continuously recognized. This is only an example for illustration, and this disclosure does not limit it.

[0092] In another possible implementation of this step, the phase information corresponding to the ultrasonic echo signal of a preset number of frames can be input into a pre-trained neural network model to obtain the model output; then, the model output can be used to determine whether the target action event has occurred.

[0093] The model output can include preset identifiers representing different events. For example, if the model output is 1, it indicates that the first event has occurred, where an obstacle approaches the terminal; if the model output is -1, it indicates that the second event has occurred, where an obstacle moves away from the terminal; if the model output is 0, it indicates that neither the first nor the second event has occurred, meaning that the relative position between the terminal and the obstacle has not changed.

[0094] In addition, when pre-training the neural network model, the feature information of the ultrasonic echo signal (including phase information, volume information, frequency shift information, etc.) can be used as training samples for model training. The specific model training process can be found in the description in the relevant literature, and is not limited here.

[0095] In step S103, if the target action event is determined to have occurred, a preset operation is executed, and different target action events correspond to different preset operations.

[0096] If the terminal's anti-mistouch function is implemented based on the action recognition result, this step can control the terminal to disable touch operation when the first event is determined to have occurred; control the terminal to enable touch operation when the second event is determined to have occurred; and control the terminal to maintain the state of the previous moment when neither the first event nor the second event is determined to have occurred.

[0097] For example, when the terminal is powered on, the default initial state of touch operation is enabled (e.g., screen on). If no action event is detected, the terminal maintains the default enabled state of touch operation. When the terminal detects the first event of an obstacle approaching the terminal, it controls the terminal to disable the touch operation to prevent accidental touches. When the terminal detects the second event of an obstacle moving away from the terminal, it controls the terminal to enable the touch operation. If no action event is detected, the terminal can maintain the enabled state of touch operation, and so on. This allows the terminal to execute preset operations based on the action recognition results. This is just an example and is not intended to limit the scope of the invention.

[0098] In another application scenario, the terminal can be controlled to execute preset functions based on the action recognition result. For example, taking a smart speaker as an example, if someone is detected approaching the speaker, the smart speaker can be controlled to turn on; if someone is detected moving away from the speaker, the smart speaker can be controlled to turn off. Therefore, this step can control the terminal to turn on the preset function when the first event is determined to have occurred; and control the terminal to turn off the preset function when the second event is determined to have occurred. The preset function can be set arbitrarily according to actual needs, such as playing / turning off preset audio, controlling the terminal to turn on / off, etc. This disclosure does not limit this.

[0099] Using the above method, phase information is extracted from each frame of received ultrasonic echo signal. Based on this phase information, it can be determined whether a first event (obstacle approaching the terminal) or a second event (obstacle moving away from the terminal) has occurred. The phase information describes the phase change of the ultrasonic echo signal, and the phase change is only related to the distance change, and is independent of the speed and the frequency response of the terminal itself. Furthermore, the phase feature is highly sensitive, meaning that a small distance change can correspond to a large change in the phase feature. Therefore, based on the phase information of the ultrasonic echo signal, the action of an obstacle approaching / moving away from the terminal can be identified more accurately, and the robustness of the identified action is better. Consequently, the terminal can be controlled to perform corresponding actions more accurately and stably based on the identification results, thereby realizing the terminal's anti-accidental touch function.

[0100] In another possible implementation of this disclosure, in order to further improve the accuracy of proximity / distance action recognition, action recognition can also be performed by combining the volume information and / or frequency shift information of the ultrasonic echo signal on the basis of phase information.

[0101] Therefore, the feature information may also include at least one of volume information and frequency shift information. The frequency shift information includes a first frequency shift parameter and a second frequency shift parameter. The first frequency shift parameter is a parameter extracted from the spectrum of the ultrasonic wave that represents a shift of the spectrum towards higher frequencies. The second frequency shift parameter is a parameter extracted from the spectrum of the ultrasonic wave that represents a shift of the spectrum towards lower frequencies. The first frequency shift parameter can be used to describe an event in which an obstacle approaches the terminal, and the second frequency shift parameter can be used to describe an event in which an obstacle moves away from the terminal.

[0102] Figure 4 This is a flowchart illustrating a method for recognizing obstacle actions according to an exemplary embodiment, such as... Figure 4 As shown, the frequency shift information in the feature information can be obtained by extracting features from this frame of ultrasonic echo signal through the following steps:

[0103] In step S1011, the first transmitted wave energy corresponding to the highest frequency of the transmitted wave and the first energy of the first other signals are obtained according to the spectrum of the ultrasonic echo signal of the frame. The first other signals include a preset number of signals with frequencies higher than the highest frequency of the transmitted wave.

[0104] As mentioned above, this disclosure can generate multiple ultrasonic signals of different frequencies using an ultrasonic generator. After being converted into analog signals by a DAC (Digital-to-Analog Converter), these signals are emitted outward by a sound wave transmitter. Assuming the frequency range of the ultrasonic signals emitted by the sound wave transmitter is 50K→200K, and the preset number is 3, the highest frequency of the emitted wave is the frequency corresponding to the 200K ultrasonic echo signal. The first other signal can include other signals (such as noise) with frequencies of 205K, 220K, and 235K on the spectrum. The energy of the first emitted wave and the first energy can both be obtained from... Figure 5 The spectrum shown can be read directly from the spectrum.

[0105] In step S1012, the first frequency shift parameter is calculated based on the first transmitted wave energy and the first energy.

[0106] In one possible implementation, the first transmitted wave energy and the first energy can be weighted and fused to obtain the first frequency shift parameter.

[0107] In step S1013, the second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and the second energy of the second other signals are obtained according to the spectrum of the ultrasonic echo signal of the frame. The second other signals include a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave.

[0108] Assuming the ultrasonic signal emitted by the sound wave transmitter has a frequency range of 50K→200K, and the preset quantity is 3, the lowest frequency of the emitted wave is the frequency corresponding to the 50K ultrasonic echo signal. The second other signal can include other signals (such as noise) with frequencies of 45K, 30K, and 15K on the spectrum. The energy of the second emitted wave and the second energy can both be obtained from, for example... Figure 5 The spectrum shown can be read directly from the spectrum.

[0109] In step S1014, the second frequency shift parameter is calculated based on the second transmitted wave energy and the second energy.

[0110] In one possible implementation, the second transmitted wave energy and the second energy can be weighted and fused to obtain the second frequency shift parameter.

[0111] In another embodiment of this disclosure, the frequency shift information can also be extracted from the spectrum of the ultrasonic echo signal in the following manner:

[0112] A frequency shift spectrum can be constructed by selecting n frequency components above and below the transmitted signal frequency (e.g., 50K→200K) from the spectrum, and this frequency shift spectrum can be used as the extracted frequency shift information for subsequent obstacle action recognition.

[0113] One approach may omit the transmitted signal frequency and instead use n frequency components above and below it to form a frequency shift spectrum. Another approach may include the transmitted signal frequency and use 2n+1 consecutive frequency components to form a frequency shift spectrum, where n can be 1, 2, 3, ...

[0114] The following describes the specific implementation method for extracting the volume information of each frame of ultrasonic echo signal:

[0115] In one possible implementation, the volume information of the ultrasonic echo signal in this frame can be extracted in the following way:

[0116] First, the time-domain signal of the ultrasonic echo signal is converted into a frequency-domain signal. Then, the energy of each frequency component is calculated based on the frequency-domain signal, as shown in the following formula:

[0117] (i) = abs(F(i)), i = 1, 2, 3, ..., n

[0118] Where F(i) represents the frequency domain signal of the i-th frequency component, (i) represents the energy of the i-th frequency component.

[0119] Then, the volume Fpow of the received ultrasonic echo signal at the current moment can be calculated using Passevar's theorem:

[0120] Fpow = RMS(F(1….n))

[0121] This is merely an example; the present disclosure may also extract volume features using methods described in other relevant literature for extracting volume features of ultrasonic signals, and the present disclosure does not limit this to such methods.

[0122] In this way, for each frame of received ultrasonic echo signal, the phase information, volume information and frequency shift information of that frame of ultrasonic echo signal can be extracted. Based on the phase information, at least one of the other two types of feature information can be combined to identify obstacle movements.

[0123] Therefore, in this disclosure, it can be determined whether the target action event has occurred based on the phase information and preset information, wherein the preset information includes at least one of the volume information and the frequency shift information.

[0124] Figure 6 This is a flowchart illustrating a method for recognizing obstacle actions according to an exemplary embodiment, such as... Figure 6 As shown, the occurrence of the target action event can be determined based on the phase information and preset information using the following steps:

[0125] In step S601, if it is determined that the phase information satisfies the first preset condition and the preset information satisfies the second preset condition, the first event is determined to have occurred.

[0126] In step S602, if it is determined that the phase information satisfies the third preset condition and the preset information satisfies the fourth preset condition, the second event is determined to have occurred.

[0127] The phase information can be determined to meet the first or third preset condition in the following ways:

[0128] The phase information corresponding to multiple frames of ultrasound echo signals received within a first preset time period is obtained; the phase difference between a first phase and a second phase is calculated, wherein the first phase includes the phase information corresponding to the last frame of ultrasound echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasound echo signal within the first preset time period; if the phase difference is greater than or equal to a first preset threshold, the phase information is determined to satisfy the first preset condition; if the phase difference is less than or equal to a second preset threshold, the phase information is determined to satisfy the third preset condition.

[0129] For a specific embodiment of determining whether the phase information meets the first preset condition or the third preset condition, please refer to [the relevant documentation]. Figure 1 The specific implementation method for identifying whether the target action event has occurred based on phase information, as shown, will not be described in detail here.

[0130] In addition, the second preset condition includes a first volume condition and / or a first frequency shift condition; the fourth preset condition includes a second volume condition and / or a second frequency shift condition; whether the preset information satisfies the second preset condition or the fourth preset condition can be determined in the following ways:

[0131] If the volume information satisfies the first volume condition and / or the frequency shift information satisfies the first frequency shift condition, then the preset information is determined to satisfy the second preset condition; if the volume information satisfies the second volume condition and / or the frequency shift information satisfies the second frequency shift condition, then the preset information is determined to satisfy the fourth preset condition.

[0132] The volume and frequency shift conditions that need to be met are analyzed below:

[0133] Figure 7 This is a volume characteristic curve of an ultrasonic echo signal illustrated according to an exemplary embodiment, from... Figure 7 As shown in the volume characteristic curve diagram of the obstacle approaching and moving away from the terminal, when the obstacle approaches the terminal, the volume characteristic shows an oscillating upward trend, and when the obstacle moves away from the terminal, the volume characteristic shows an oscillating downward trend. Therefore, this disclosure can determine whether the volume information meets the first volume condition or the second volume condition through any of the following three methods:

[0134] Method 1: If the volume information meets the preset volume sub-condition, calculate the volume change slope of the ultrasonic echo signal within the second preset time period based on the volume information. If the volume change slope is positive, determine that the volume information meets the first volume condition. If the volume change slope is negative, determine that the volume information meets the second volume condition.

[0135] Wherein, the first volume condition is the volume condition corresponding to the occurrence of the first event, the second volume condition is the volume condition corresponding to the occurrence of the second event, and the preset volume sub-condition includes:

[0136] The volume variance is greater than or equal to a third preset threshold, which is the variance corresponding to the volume information of the ultrasonic echo signal received within a second preset time period; or, the sum of the peak-to-trough differences is greater than or equal to a fourth preset threshold, which is the sum of the peak-to-trough differences of the volume curve of the ultrasonic echo signal received within the second preset time period; or, the difference between the maximum volume and the minimum volume is greater than or equal to a fifth preset threshold, which is the difference between the maximum volume and the minimum volume of the ultrasonic echo signal received within the second preset time period.

[0137] It is understandable that, given that the volume information satisfies the aforementioned preset volume sub-condition, it can be determined that the target action event may occur, but it is still impossible to determine whether the target action event is the first event or the second event. Therefore, it is possible to further determine whether the volume information satisfies the first volume condition or the second volume condition based on the sign of the slope of the volume change of the ultrasonic echo signal within the second preset time period, such as... Figure 7 As shown, when the slope of the volume change is positive, it indicates that the volume of the ultrasonic echo signal increases at a certain slope, and at this time, the volume information is determined to meet the first volume condition. When the slope of the volume change is negative, it indicates that the volume of the ultrasonic echo signal decreases at a certain slope, and at this time, the volume information is determined to meet the second volume condition.

[0138] Method 2: If the volume information meets the preset volume sub-condition, determine the changing trend of the volume peak on the volume curve within the second preset time period. If the changing trend is increasing, determine that the volume information meets the first volume condition. If the changing trend is decreasing, determine that the volume information meets the second volume condition.

[0139] Method 3: If the volume information meets the preset volume sub-condition, perform a linear function fitting on the volume peak to obtain the fitting function; if the slope of the fitting function is positive, determine that the volume information meets the first volume condition; if the slope of the fitting function is negative, determine that the volume information meets the second volume condition.

[0140] Method 2 determines whether the volume information meets the preset volume conditions based on the changing trend of the volume peak on the volume curve within the second preset time period. Method 3, on the other hand, performs a linear function fitting on the volume peak to obtain a fitting function, and determines whether the volume information meets the preset volume conditions based on the positive or negative slope of the fitting function. In essence, both methods determine whether the volume changes within the second preset time period by increasing or decreasing, thereby determining whether the volume information meets the first volume condition or the second volume condition.

[0141] This disclosure can determine whether the frequency shift information satisfies the first frequency shift condition or the second frequency shift condition in the following ways:

[0142] As mentioned above, the first frequency shift parameter characterizes the frequency spectrum shift towards higher frequencies, and the second frequency shift parameter characterizes the frequency spectrum shift towards lower frequencies. Based on the Doppler effect, it is known that when the terminal is near an obstacle, the frequency of the ultrasonic echo signal received by the terminal increases, and when the terminal is far from an obstacle, the frequency of the ultrasonic echo signal received by the terminal decreases. Therefore, when the first frequency shift parameter is greater than or equal to a sixth preset threshold, and the first frequency shift parameter is greater than or equal to K times the second frequency shift parameter, the frequency shift information is determined to satisfy the first frequency shift condition, where K is a preset number. When the second frequency shift parameter is greater than or equal to a seventh preset threshold, and the second frequency shift parameter is greater than or equal to K times the first frequency shift parameter, the frequency shift information is determined to satisfy the second frequency shift condition.

[0143] Based on the above threshold determination method, for the ultrasonic echo signal received within a certain period of time, obstacle action can be identified according to the extracted phase information, volume information and frequency shift information, thereby improving the accuracy of identification. In addition, this disclosure can also identify action based on only one of the volume information and frequency shift information of the ultrasonic echo signal.

[0144] For example, in one approach, if the volume information satisfies a first volume condition, the first event is determined to occur; if the volume information satisfies a second volume condition, the second event is determined to occur. In another approach, if the frequency shift information satisfies a first frequency shift condition, the first event is determined to occur; if the frequency shift information satisfies a second frequency shift condition, the second event is determined to occur.

[0145] In addition to the threshold determination method, this disclosure can also input the phase information corresponding to the ultrasonic echo signal of a preset number of frames and the preset information into a pre-trained neural network model to obtain the model output; and determine whether the target action event has occurred based on the model output.

[0146] The specific implementation here is similar to the process of action recognition based on phase information through a neural network model, and will not be elaborated further here.

[0147] It should be noted that the feature information mentioned above can be a primary feature. Secondary operations (such as variance calculation, difference calculation, etc.) or other forms of transformation based on the primary feature are all feature information used in this disclosure to identify the actions of obstacles approaching / moving away from the terminal.

[0148] Using the above method, it is possible to determine whether a first event (obstacle approaching the terminal) or a second event (obstacle moving away from the terminal) has occurred based on the phase information of the received ultrasonic echo signal. The phase information describes the phase change of the ultrasonic echo signal, and the phase change is only related to the distance change, not to the speed or the frequency response of the terminal itself. Furthermore, the phase feature is highly sensitive, meaning that a small distance change can correspond to a large change in the phase feature. Therefore, based on the phase information of the ultrasonic echo signal, the action of an obstacle approaching / moving away from the terminal can be identified more accurately, and the robustness of the identified action is better. Consequently, the terminal can be controlled to perform corresponding actions more accurately and stably based on the identification results, realizing the terminal's anti-accidental touch function. At the same time, this disclosure uses a sound wave transmitter to emit ultrasonic signals of multiple frequencies, which has stronger anti-interference capabilities compared to the related technology of emitting a single frequency ultrasonic signal for obstacle action identification.

[0149] In addition, considering that the process of identifying the approach / remote action through the feature information of the ultrasonic echo signal only requires the transmission and reception of the signal through the terminal's inherent sound-generating device (such as a speaker, microphone, etc.) and recording device (such as a microphone), the ultrasonic generator, feature extraction and action recognition are all implemented through software algorithms, without occupying additional device space, thus saving hardware costs.

[0150] Figure 8 This is a block diagram illustrating a device for recognizing obstacle movements according to an exemplary embodiment, such as... Figure 8 As shown, the device includes:

[0151] The feature extraction module 801 is configured to extract feature information from each received frame of ultrasonic echo signal to obtain feature information, the feature information including phase information; the ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal.

[0152] The first determining module 802 is configured to determine whether a target action event has occurred based on the phase information. The target action event includes a first event in which an obstacle approaches the terminal or a second event in which the obstacle moves away from the terminal.

[0153] The action execution module 803 is configured to perform a preset operation when the target action event is determined to have occurred, with different preset operations corresponding to different target action events.

[0154] Optionally, the feature extraction module 801 is configured to perform frequency domain transformation on the frame of ultrasonic echo signal to obtain a frequency domain signal; and extract the phase information corresponding to the frame of ultrasonic echo signal based on the real and imaginary parts of the frequency domain signal.

[0155] Optionally, the first determining module 802 is configured to acquire the phase information corresponding to multiple frames of ultrasonic echo signals received within a first preset time period; calculate the phase difference between a first phase and a second phase, wherein the first phase includes the phase information corresponding to the last frame of ultrasonic echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasonic echo signal within the first preset time period; determine that the first event has occurred if the phase difference is greater than or equal to a first preset threshold; and determine that the second event has occurred if the phase difference is less than or equal to a second preset threshold.

[0156] Optionally, the first determining module 802 is configured to input the phase information corresponding to the ultrasonic echo signals of a preset number of frames into a pre-trained neural network model to obtain the model output; and determine whether the target action event has occurred based on the model output.

[0157] Optionally, the feature information further includes at least one of volume information and frequency shift information. The frequency shift information includes a first frequency shift parameter and a second frequency shift parameter. The feature extraction module 801 is configured to obtain, based on the spectrum of the ultrasonic echo signal of the frame, a first transmitted wave energy corresponding to the highest frequency of the transmitted wave and a first energy of a first other signal, wherein the first other signal includes a preset number of signals with frequencies higher than the highest frequency of the transmitted wave; calculate the first frequency shift parameter based on the first transmitted wave energy and the first energy; obtain, based on the spectrum of the ultrasonic echo signal of the frame, a second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and a second energy of a second other signal, wherein the second other signal includes a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave; and calculate the second frequency shift parameter based on the second transmitted wave energy and the second energy.

[0158] Optionally, the first determining module 802 is configured to determine whether the target action event has occurred based on the phase information and preset information, wherein the preset information includes at least one of the volume information and the frequency shift information.

[0159] Optionally, the first determining module 802 is configured to determine that the first event has occurred when the phase information satisfies a first preset condition and the preset information satisfies a second preset condition; and to determine that the second event has occurred when the phase information satisfies a third preset condition and the preset information satisfies a fourth preset condition.

[0160] Optionally, the first determining module 802 is configured to acquire the phase information corresponding to multiple frames of ultrasonic echo signals received within a first preset time period; calculate the phase difference between a first phase and a second phase, wherein the first phase includes the phase information corresponding to the last frame of ultrasonic echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of ultrasonic echo signal within the first preset time period; if the phase difference is greater than or equal to a first preset threshold, determine that the phase information satisfies the first preset condition; if the phase difference is less than or equal to a second preset threshold, determine that the phase information satisfies the third preset condition.

[0161] Optionally, the second preset condition includes a first volume condition and / or a first frequency shift condition; the fourth preset condition includes a second volume condition and / or a second frequency shift condition; the first determining module 802 is configured to determine that the preset information satisfies the second preset condition when the volume information satisfies the first volume condition and / or the frequency shift information satisfies the first frequency shift condition; and to determine that the preset information satisfies the fourth preset condition when the volume information satisfies the second volume condition and / or the frequency shift information satisfies the second frequency shift condition.

[0162] Specifically, when the volume information meets a preset volume sub-condition, the volume change slope of the ultrasonic echo signal within a second preset time period is calculated based on the volume information. If the volume change slope is positive, the volume information is determined to meet the first volume condition. If the volume change slope is negative, the volume information is determined to meet the second volume condition.

[0163] or,

[0164] If the volume information meets a preset volume sub-condition, determine the trend of the volume peak value on the volume curve within the second preset time period. If the trend of ...

[0165] or,

[0166] If the volume information satisfies a preset volume sub-condition, a linear function is performed on the volume peak to obtain a fitting function; if the slope of the fitting function is positive, the volume information is determined to satisfy the first volume condition; if the slope of the fitting function is negative, the volume information is determined to satisfy the second volume condition.

[0167] The preset volume sub-conditions include:

[0168] The volume variance is greater than or equal to a third preset threshold, where the volume variance is the variance corresponding to the volume information of the ultrasonic echo signal received within a second preset time period; or, the sum of the peak-to-trough differences is greater than or equal to a fourth preset threshold, where the sum of the peak-to-trough differences is the sum of the peak-to-trough differences of the volume curve of the ultrasonic echo signal received within the second preset time period; or, the difference between the maximum volume and the minimum volume is greater than or equal to a fifth preset threshold, where the difference between the maximum volume and the minimum volume is the difference between the maximum volume and the minimum volume of the ultrasonic echo signal received within the second preset time period.

[0169] If the first frequency shift parameter is greater than or equal to a sixth preset threshold, and the first frequency shift parameter is greater than or equal to K times the second frequency shift parameter, then the frequency shift information is determined to satisfy the first frequency shift condition, where K is a preset number.

[0170] If the second frequency shift parameter is greater than or equal to a seventh preset threshold, and the second frequency shift parameter is greater than or equal to the first frequency shift parameter multiplied by K, then the frequency shift information is determined to satisfy the second frequency shift condition.

[0171] Optionally, the first determining module 802 is configured to: input the phase information corresponding to the ultrasonic echo signal of a preset number of frames and the preset information into a pre-trained neural network model to obtain the model output; and determine whether the target action event has occurred based on the model output.

[0172] Optionally, the action execution module 803 is configured to: control the terminal to disable touch operation when the first event is determined to occur; and control the terminal to enable touch operation when the second event is determined to occur.

[0173] Optionally, the action execution module 803 is configured to: control the terminal to enable a preset function when the first event is determined to occur; and control the terminal to disable the preset function when the second event is determined to occur.

[0174] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0175] Using the aforementioned device, phase information is obtained by extracting features from each received frame of ultrasonic echo signal. Based on this phase information, it can be determined whether a first event (obstacle approaching the terminal) or a second event (obstacle moving away from the terminal) has occurred. The phase information describes the phase change of the ultrasonic echo signal, and the phase change is only related to the distance change, and is independent of the speed and the frequency response of the terminal itself. Furthermore, the phase feature is highly sensitive, meaning that a small distance change can correspond to a large change in the phase feature. Therefore, based on the phase information of the ultrasonic echo signal, the action of an obstacle approaching / moving away from the terminal can be identified more accurately, and the robustness of the identified action is better. Consequently, the terminal can be controlled to perform corresponding actions more accurately and stably based on the identification results, thereby realizing the terminal's anti-accidental touch function.

[0176] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the method for recognizing obstacle actions provided in this disclosure.

[0177] Figure 9 This is a block diagram illustrating a device 900 for recognizing obstacle movements according to an exemplary embodiment. The device 900 can be an electronic device, such as a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0178] Reference Figure 9 The device 900 may include one or more of the following components: a processing component 902, a memory 904, a power component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0179] Processing component 902 typically controls the overall operation of device 900, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the method for recognizing obstacles described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.

[0180] Memory 904 is configured to store various types of data to support the operation of device 900. Examples of this data include instructions for any application or method operating on device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0181] The power supply component 906 provides power to the various components of the device 900. The power supply component 906 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 900.

[0182] Multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0183] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when device 900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.

[0184] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0185] Sensor assembly 914 includes one or more sensors for providing status assessments of various aspects of device 900. For example, sensor assembly 914 may detect the on / off state of device 900, the relative positioning of components such as the display and keypad of device 900, changes in position of device 900 or a component of device 900, the presence or absence of user contact with device 900, orientation or acceleration / deceleration of device 900, and temperature changes of device 900. Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0186] Communication component 916 is configured to facilitate wired or wireless communication between device 900 and other devices. Device 900 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0187] In an exemplary embodiment, the device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method for recognizing obstacles.

[0188] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of the device 900 to complete the method for recognizing obstacles described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0189] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described method of recognizing obstacles when executed by the programmable device.

[0190] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0191] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for recognizing obstacle movements, characterized in that, include: For each received frame of ultrasonic echo signal, feature extraction is performed on the frame of ultrasonic echo signal to obtain feature information, which includes phase information; The ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal; Based on the phase information, it is determined whether a target action event has occurred, the target action event including a first event of an obstacle approaching the terminal or a second event of the obstacle moving away from the terminal; When the target action event is determined to have occurred, a preset operation is executed, with different preset operations corresponding to different target action events; The feature information further includes at least one of volume information and frequency shift information, wherein the frequency shift information includes a first frequency shift parameter and a second frequency shift parameter, and the feature extraction of the ultrasonic echo signal frame to obtain the feature information includes: The first transmitted wave energy corresponding to the highest frequency of the transmitted wave and the first energy of the first other signals are obtained based on the spectrum of the ultrasonic echo signal of the frame. The first other signals include a preset number of signals with frequencies higher than the highest frequency of the transmitted wave. The first frequency shift parameter is calculated based on the first transmitted wave energy and the first energy. The second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and the second energy of the second other signals are obtained from the spectrum of the ultrasonic echo signal of the frame. The second other signals include a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave. The second frequency shift parameter is calculated based on the second transmitted wave energy and the second energy.

2. The method according to claim 1, characterized in that, The feature information obtained by extracting features from the ultrasonic echo signal of that frame includes: The ultrasonic echo signal in this frame is transformed in the frequency domain to obtain the frequency domain signal; The phase information corresponding to the ultrasonic echo signal of the frame is extracted based on the real and imaginary parts of the frequency domain signal.

3. The method according to claim 1, characterized in that, The step of determining whether a target action event has occurred based on the phase information includes: Obtain the phase information corresponding to the multiple frames of ultrasonic echo signals received within a first preset time period; Calculate the phase difference between the first phase and the second phase, wherein the first phase includes the phase information corresponding to the last frame of the ultrasound echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of the ultrasound echo signal within the first preset time period; If the phase difference is greater than or equal to a first preset threshold, the occurrence of the first event is determined. If the phase difference is less than or equal to a second preset threshold, the second event is determined to have occurred.

4. The method according to claim 1, characterized in that, The step of determining whether a target action event has occurred based on the phase information includes: The phase information corresponding to the ultrasonic echo signals of a preset number of frames is input into a pre-trained neural network model to obtain the model output; The model output determines whether the target action event has occurred.

5. The method according to claim 1, characterized in that, The step of determining whether a target action event has occurred based on the phase information includes: The target action event is determined to have occurred based on the phase information and preset information, wherein the preset information includes at least one of the volume information and the frequency shift information.

6. The method according to claim 5, characterized in that, The step of determining whether the target action event has occurred based on the phase information and preset information includes: If the phase information satisfies a first preset condition and the preset information satisfies a second preset condition, then the first event is determined to have occurred. If the phase information satisfies a third preset condition and the preset information satisfies a fourth preset condition, then the second event is determined to have occurred.

7. The method according to claim 6, characterized in that, The method includes: Obtain the phase information corresponding to the multiple frames of ultrasonic echo signals received within a first preset time period; Calculate the phase difference between the first phase and the second phase, wherein the first phase includes the phase information corresponding to the last frame of the ultrasound echo signal within the first preset time period, and the second phase includes the phase information corresponding to the first frame of the ultrasound echo signal within the first preset time period; If the phase difference is greater than or equal to a first preset threshold, the phase information is determined to satisfy the first preset condition; if the phase difference is less than or equal to a second preset threshold, the phase information is determined to satisfy the third preset condition.

8. The method according to claim 6, characterized in that, The second preset condition includes a first volume condition and / or a first frequency shift condition; The fourth preset condition includes a second volume condition and / or a second frequency shift condition; the method includes: If the volume information satisfies the first volume condition and / or the frequency shift information satisfies the first frequency shift condition, then the preset information is determined to satisfy the second preset condition. If the volume information satisfies the second volume condition, and / or the frequency shift information satisfies the second frequency shift condition, then the preset information is determined to satisfy the fourth preset condition. Determining whether the volume information meets the first volume condition or the second volume condition includes: If the volume information satisfies a preset volume sub-condition, the volume change slope of the ultrasonic echo signal within a second preset time period is calculated based on the volume information. If the volume change slope is positive, the volume information is determined to satisfy the first volume condition. If the volume change slope is negative, the volume information is determined to satisfy the second volume condition. or, If the volume information meets a preset volume sub-condition, determine the trend of the volume peak value on the volume curve within the second preset time period. If the trend of ... or, If the volume information satisfies a preset volume sub-condition, a linear function is performed on the volume peak to obtain a fitting function; if the slope of the fitting function is positive, the volume information is determined to satisfy the first volume condition; if the slope of the fitting function is negative, the volume information is determined to satisfy the second volume condition. The preset volume sub-conditions include: The volume variance is greater than or equal to a third preset threshold, where the volume variance is the variance corresponding to the volume information of the ultrasonic echo signal received within a second preset time period; or, the sum of the peak-to-trough differences is greater than or equal to a fourth preset threshold, where the sum of the peak-to-trough differences is the sum of the peak-to-trough differences of the volume curve of the ultrasonic echo signal received within the second preset time period; or, the difference between the maximum volume and the minimum volume is greater than or equal to a fifth preset threshold, where the difference between the maximum volume and the minimum volume is the difference between the maximum volume and the minimum volume of the ultrasonic echo signal received within the second preset time period. Determining whether the frequency shift information satisfies the first frequency shift condition or the second frequency shift condition includes: If the first frequency shift parameter is greater than or equal to a sixth preset threshold, and the first frequency shift parameter is greater than or equal to K times the second frequency shift parameter, then the frequency shift information is determined to satisfy the first frequency shift condition, where K is a preset number. If the second frequency shift parameter is greater than or equal to a seventh preset threshold, and the second frequency shift parameter is greater than or equal to the first frequency shift parameter multiplied by K, then the frequency shift information is determined to satisfy the second frequency shift condition.

9. The method according to claim 5, characterized in that, The step of determining whether the target action event has occurred based on the phase information and preset information includes: The phase information corresponding to the ultrasonic echo signal of a preset number of frames and the preset information are input into a pre-trained neural network model to obtain the model output. The model output determines whether the target action event has occurred.

10. The method according to any one of claims 1-9, characterized in that, The step of performing a preset operation upon determining that the target action event has occurred includes: If the occurrence of the first event is determined, the terminal is controlled to disable touch operation; If the second event is determined to have occurred, the terminal is controlled to enable touch operation.

11. The method according to any one of claims 1-9, characterized in that, The step of performing a preset operation upon determining that the target action event has occurred includes: If the occurrence of the first event is confirmed, the terminal is controlled to activate a preset function; If the second event is determined to have occurred, the terminal is controlled to disable the preset function.

12. A device for recognizing the movement of obstacles, characterized in that, include: The feature extraction module is configured to extract feature information from each received frame of ultrasonic echo signal to obtain feature information, including phase information; the ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal. The first determining module is configured to determine whether a target action event has occurred based on the phase information. The target action event includes a first event in which an obstacle approaches the terminal or a second event in which the obstacle moves away from the terminal. The action execution module is configured to perform a preset operation when the target action event is determined to have occurred, with different preset operations corresponding to different target action events; The feature information further includes at least one of volume information and frequency shift information. The frequency shift information includes a first frequency shift parameter and a second frequency shift parameter. The feature extraction module is configured to obtain, based on the spectrum of the ultrasonic echo signal frame, a first transmitted wave energy corresponding to the highest frequency of the transmitted wave and a first energy of a first other signal, wherein the first other signal includes a preset number of signals with frequencies higher than the highest frequency of the transmitted wave; calculate the first frequency shift parameter based on the first transmitted wave energy and the first energy; obtain, based on the spectrum of the ultrasonic echo signal frame, a second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and a second energy of a second other signal, wherein the second other signal includes a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave; and calculate the second frequency shift parameter based on the second transmitted wave energy and the second energy.

13. A device for recognizing the movement of obstacles, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to: extract feature information from each received frame of ultrasonic echo signal to obtain feature information, the feature information including phase information; the ultrasonic echo signal is the reflected signal of the emitted ultrasonic signal; Based on the phase information, it is determined whether a target action event has occurred, the target action event including a first event of an obstacle approaching the terminal or a second event of the obstacle moving away from the terminal; When the target action event is determined to have occurred, a preset operation is executed, with different preset operations corresponding to different target action events; The feature information further includes at least one of volume information and frequency shift information, wherein the frequency shift information includes a first frequency shift parameter and a second frequency shift parameter, and the feature extraction of the ultrasonic echo signal frame to obtain the feature information includes: The first transmitted wave energy corresponding to the highest frequency of the transmitted wave and the first energy of the first other signals are obtained based on the spectrum of the ultrasonic echo signal of the frame. The first other signals include a preset number of signals with frequencies higher than the highest frequency of the transmitted wave. The first frequency shift parameter is calculated based on the first transmitted wave energy and the first energy. The second transmitted wave energy corresponding to the lowest frequency of the transmitted wave and the second energy of the second other signals are obtained from the spectrum of the ultrasonic echo signal of the frame. The second other signals include a preset number of signals with frequencies lower than the lowest frequency of the transmitted wave. The second frequency shift parameter is calculated based on the second transmitted wave energy and the second energy.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 11.

Images