Ultrasonic proximity detection method and system, electronic equipment, medium and program product
By frequency separation and classification of ultrasonic waves, the status feature categories are summarized to determine the final state of the terminal equipment, the problems of low accuracy and poor migration ability in the prior art are solved, and higher detection accuracy and computing efficiency are achieved.
Patent Information
- Application Number
- CN202410969373.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
The existing ultrasonic proximity detection technology has problems such as low accuracy, poor classifier migration ability, and large calculation amount. Especially under the overall judgment of multi-frequency, it is difficult to effectively distinguish the status of the terminal equipment, resulting in low accuracy of algorithm judgment and difficulty in adapting to new equipment. the frequency distribution changes.
By separating the ultrasonic transmitting signals into several different frequencies, the frequency separation and classification of the received signals are performed separately, the state feature categories are summarized to determine the final state characteristics of the terminal equipment, and the detection accuracy and migration ability are improved using confidence weighting or voting methods.
It improves the accuracy of ultrasonic proximity detection, reduces the calculation amount of classifiers, enhances the adaptability to the frequency distribution of new devices, and reduces the need for retraining and parameter adjustment.
Smart Images

Figure CN118944694A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of electronic devices, and in particular to an ultrasonic proximity detection method, system, electronic device, medium, and program product. Background Art
[0002] Ultrasonic proximity detection is used in applications such as screen-off during calls (screen off when close to the ear, screen on when away from the ear), gesture recognition (for example, close to the next page, away from the previous page, or close to increase the volume, away from the volume), and more. The terminal can be a mobile phone, tablet, watch, etc. This function requires the terminal's speaker to emit ultrasonic waves, and then uses the terminal's microphone to receive the direct ultrasonic wave and reflected echo. The characteristics of the received signal (primarily the Doppler frequency shift) are analyzed to achieve proximity detection.
[0003] Existing solutions treat all frequencies as a whole ("multi-frequency holistic judgment"). For example, after preprocessing the signal frequency using the STFT (Short-Time Fourier Transform), a slice of the time-frequency graph containing all transmitted frequency intervals is cut off and input into the classifier. However, the following problems may arise:
[0004] First, the classifier for received signal features uses a two-category output based on the state. This results in two difficult-to-distinguish scenarios (near and far) being classified into two different categories, hindering algorithm judgment. For example, if the screen-off feature during a call is used as an example, if the classifier for received signal features uses a two-category output based on the screen state (on / off), two difficult-to-distinguish scenarios (the phone is held to the ear and far away) are classified into two different categories, hindering algorithm judgment and reducing algorithm accuracy.
[0005] Second, the transmitted signal is a mixture of multiple single-frequency signals, and the classifier judges all frequencies as a whole ("multi-frequency overall judgment"). If an AI (artificial intelligence) model is used as the classifier, the required model size is large, and the classifier's migration capabilities are insufficient. That is, when training or adjusting the classifier for multi-frequency overall judgment, the new device must use a frequency distribution different from the original data due to factors such as frequency response and interference. In this case, the classifier for multi-frequency overall judgment often cannot be used directly, but instead requires new data to be collected with the new transmission frequency distribution, and retraining or parameter adjustment is required. This process is time-consuming and labor-intensive, and will slow down the application of the algorithm to new devices.
[0006] Third, when using existing gesture recognition solutions, defining a variety of very different gestures and realizing a variety of different functions will make the classifier more complex and the amount of calculations large. Summary of the Invention
[0007] The technical problem to be solved by the present disclosure is to overcome the defects of the prior art, such as low proximity detection accuracy, poor classifier migration capability, and large computational complexity, and to provide an ultrasonic proximity detection method, system, electronic device, medium, and program product.
[0008] The present disclosure solves the above technical problems through the following technical solutions:
[0009] The present disclosure provides an ultrasonic proximity detection method, which is applied to a terminal device. The ultrasonic proximity detection method includes:
[0010] Receive ultrasonic waves; wherein the ultrasonic waves are formed by superimposing a plurality of transmission signals of different transmission frequencies;
[0011] Performing frequency separation on the ultrasonic wave to obtain a received signal corresponding to the transmitted signal;
[0012] Input each received signal into a classifier to output a corresponding state feature category of the terminal device; wherein the state feature category includes a stationary feature, an approaching feature, and a distant feature;
[0013] Summarize the status feature categories corresponding to all received signals, and determine the final status feature of the terminal device according to the summarized status feature categories.
[0014] Preferably, the step of performing frequency separation on the ultrasonic wave comprises:
[0015] Preprocessing the ultrasonic wave to obtain a time-frequency diagram of the ultrasonic wave;
[0016] The step of performing frequency separation on the ultrasonic wave comprises:
[0017] A frequency domain slice of the time-frequency diagram is taken with each transmission frequency as the center.
[0018] Preferably, the step of performing frequency separation on the ultrasonic wave comprises:
[0019] The ultrasonic waves are subjected to frequency separation by a peak filter.
[0020] Preferably, the step of determining the final state feature of the terminal device according to the summarized state feature categories includes:
[0021] In response to the state feature category of each output corresponding to each received signal being not unique, obtaining a confidence level of each output state feature category;
[0022] Performing a weighted summation on the confidence level and the frequency of the received signal to obtain a weighted summation value corresponding to each received signal;
[0023] Determining the state feature category corresponding to the first sum value as the final state feature of the terminal device;
[0024] Wherein, the first sum value is the maximum value among the weighted sum values;
[0025] or,
[0026] In response to each received signal corresponding to the output state feature category being unique, counting the number of the state feature categories;
[0027] In response to the number of the first feature category being greater than the number of the second feature category, and the number of the first feature category being greater than the number of the third feature category, determining that the first feature category is a final state feature of the terminal device; wherein the first feature category, the second feature category, and the third feature category are any one of a stationary feature, an approaching feature, and a moving-away feature;
[0028] In response to the number of approaching features being the same as the number of moving away features, determining the stationary feature as the final state feature of the terminal device;
[0029] In response to the number of approaching features being the same as the number of stationary features, and the number of moving away features being less than the number of approaching features, determining the approaching feature as a final state feature of the terminal device;
[0030] In response to the number of the moving-away features being the same as the number of the stationary features, and the number of the approaching features being less than the number of the moving-away features, determining the moving-away feature as a final state feature of the terminal device;
[0031] or,
[0032] In response to the number of the moving-away features being greater than the number of the stationary features, and the number of the moving-away features being greater than the number of the approaching features, determining the moving-away feature as a final state feature of the terminal device;
[0033] In response to the number of approaching features being greater than the number of stationary features, and the number of approaching features being greater than the number of moving away features, determining the approaching feature as a final state feature of the terminal device;
[0034] Otherwise, the static feature is determined to be the final state feature of the terminal device;
[0035] or,
[0036] Obtaining current status characteristics of the terminal device;
[0037] In response to the current state feature being the first state, determining the sum of the number of stationary features and the old number of close features as the new number of close features;
[0038] In response to the current state feature being in the second state, determining the sum of the number of stationary features and the old number of distant features as the new number of distant features;
[0039] In response to the new number of the approaching feature being greater than the new number of the moving away feature, determining the approaching feature as a final state feature of the terminal device;
[0040] In response to the new number of the moving away feature being greater than the new number of the moving closer feature, determining that the moving away feature is a final state feature of the terminal device;
[0041] In response to the new number of the approaching feature being equal to the new number of the moving away feature, determining the stationary feature as the final state feature of the terminal device;
[0042] In response to the new number of the approaching feature being equal to the new number of the moving away feature, and the old number of the approaching feature being equal to the old number of the moving away feature, determining that the stationary feature is a final state feature of the terminal device;
[0043] In response to the new number of the approaching feature being equal to the new number of the moving away feature, and the old number of the approaching feature being greater than the old number of the moving away feature, determining that the approaching feature is a final state feature of the terminal device;
[0044] In response to the new number of the approaching feature being equal to the new number of the moving away feature, and the old number of the moving away feature being greater than the old number of the approaching feature, determining that the moving away feature is the final state feature of the terminal device.
[0045] Preferably, the ultrasonic proximity detection method further includes:
[0046] Controlling the terminal device accordingly according to the final state characteristics;
[0047] or,
[0048] In response to the final state feature of the terminal device appearing a preset number of times continuously, the terminal device is controlled accordingly.
[0049] Preferably, the step of inputting each received signal into a classifier to output a corresponding state feature category of the terminal device; wherein the state feature category includes a stationary feature, a close feature, and a far feature comprises:
[0050] In response to the frequency of the received signal being greater than the transmit frequency, outputting a proximity characteristic;
[0051] outputting a distance characteristic in response to the frequency of the received signal being less than the transmit frequency;
[0052] In response to the frequency of the received signal being equal to the transmit frequency, a stationary feature is output.
[0053] The present disclosure also provides an ultrasonic proximity detection system, which is applied to a terminal device. The ultrasonic proximity detection system includes:
[0054] A receiving module, configured to receive ultrasonic waves, wherein the ultrasonic waves are generated by superimposing a plurality of transmission signals of different transmission frequencies;
[0055] A frequency separation module, configured to perform frequency separation on the ultrasonic wave to obtain a received signal corresponding to the transmitted signal;
[0056] A feature module, configured to input each received signal into a classifier to output a corresponding state feature category of the terminal device; wherein the state feature category includes a stationary feature, an approaching feature, and a distant feature;
[0057] The determination module is used to summarize the status feature categories corresponding to all received signals and determine the final status feature of the terminal device based on the summarized status feature categories.
[0058] The present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and configured to run on the processor, wherein the processor implements the above-mentioned ultrasonic proximity detection method when executing the computer program.
[0059] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned ultrasonic proximity detection method is implemented.
[0060] The present disclosure also provides a chip, which is applied to electronic equipment and is used to execute the above-mentioned ultrasonic proximity detection method.
[0061] The present disclosure also provides a chip module, which is applied to electronic equipment, including a transceiver component and a chip, wherein the chip is used to execute the above-mentioned ultrasonic proximity detection method.
[0062] The present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned ultrasonic proximity detection method when executed by a processor.
[0063] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.
[0064] The positive progress of this disclosure is:
[0065] The present invention obtains received signals corresponding to all transmitted signals by frequency separation of received ultrasonic waves, inputs each received signal into a classifier to obtain a state feature category of a terminal device, summarizes and analyzes the state feature categories corresponding to all received signals, and determines the final state feature of the terminal device, thereby improving the ultrasonic proximity detection accuracy of the terminal device and improving the migration capability of the classifier while reducing the computational complexity of the classifier. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 The time-frequency diagram of the ideal state of ultrasonic detection of screen-off during calls is close to that of existing solutions;
[0067] Figure 2 The time-frequency diagram of the actual state of ultrasonic detection of screen-off during call close to the existing solution;
[0068] Figure 3 A flowchart of an ultrasonic proximity detection method provided in Example 1 of the present disclosure;
[0069] Figure 4 A flowchart of a specific example of an ultrasonic proximity detection method provided in Example 1 of the present disclosure;
[0070] Figure 5 A structural diagram of an ultrasonic proximity detection system provided in Example 2 of the present disclosure;
[0071] Figure 6 This is a schematic diagram of the structure of the electronic device provided in Example 3 of the present disclosure. DETAILED DESCRIPTION
[0072] The present disclosure is further illustrated below by way of examples, but the present disclosure is not limited to the scope of the examples.
[0073] In the embodiments of the present disclosure, prefixes such as "first" and "second" are used only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. In the embodiments of the present disclosure, the use of prefixes such as ordinal numbers to distinguish description objects does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitations should be constituted due to the use of such prefixes. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.
[0074] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0075] Example 1
[0076] Application scenarios for ultrasonic proximity detection in terminal devices include: screen-off during calls (screen off when close to the ear, screen on when away from the ear), gesture recognition (for example, close to the ear indicates the next page, away from the ear indicates the previous page, or close to increase the volume, away from the ear indicates the volume decrease), etc. Terminal devices include mobile phones, tablets, watches, and other devices. Ultrasonic proximity detection requires the terminal device's speaker to emit ultrasonic waves, and then the terminal device's microphone to receive the direct ultrasonic wave and reflected echo. The characteristics of the received signal (primarily the Doppler frequency shift) are analyzed to achieve ultrasonic proximity detection.
[0077] Taking the call screen off function as an example, the existing implementation scheme is to transmit ultrasonic continuous waves (transmitted signals) of multiple frequencies, detect Doppler frequency shift characteristics, and conduct a two-classification, multi-frequency joint analysis of the Doppler frequency shift characteristics. In the ideal model, the relationship between the received ultrasonic frequency (the frequency of the received signal) and the time when the phone approaches and moves away from the user's head (time-frequency diagram) is shown in the reference. Figure 1 shown. Figure 1 The horizontal axis is used to represent time. Figure 1 The vertical axis is used to represent the instantaneous frequency. Figure 1 In the received signal, there are direct waves (①②⑤⑥⑨) and reflected waves (③④⑤⑦⑧). The direct wave exists from beginning to end, and its frequency is always equal to the transmitted signal frequency f0. The amplitude of the reflected wave depends on the distance between the phone and the human ear, and the frequency depends on the relative motion between the phone and the human ear. The Doppler shift formula is Among them, f0 is the transmission frequency, v is the movement speed, c is the speed of sound, and Δf is the frequency shift. It takes a positive sign when the mobile phone is relatively close to the human ear, and a negative sign when the mobile phone is relatively close to the human ear.
[0078] At first, the phone is placed far away from the ear and can only receive direct waves① without reflected waves.
[0079] When the phone is brought close to the ear, the Doppler shift of the reflected wave is positive. This approach process includes both accelerated and decelerated approaches. During accelerated approaches, the Doppler shift of the reflected wave is positive and increases, but the phone is far from the ear at this time, so the reflected wave cannot be received with sufficient amplitude to be discerned (see ③ in the figure). During decelerated approaches, the Doppler shift of the reflected wave is positive and decreases, indicating that the phone is close enough to the ear to receive a sufficiently strong reflected wave (see ④ in the figure). Furthermore, the direct wave ② is always received.
[0080] When a phone is held close to the ear, the Doppler shift of the reflected wave is zero and the amplitude is strong. The direct and reflected waves both have a frequency of f0, and their superposition is shown as signal ⑤ on the time-frequency diagram. Due to the multipath effect, the phase difference between the direct and reflected waves is uncertain (depending on the distance between the phone and the ear). Therefore, the superimposed signal ⑤ may have a larger amplitude than the pure direct waves ①, ②, ⑥, and ⑨, or may have a smaller amplitude than or equal to the pure direct waves ①, ②, ⑥, and ⑨.
[0081] When the phone is moved away from the ear, the Doppler shift of the reflected wave is negative. This process involves both acceleration and deceleration. During acceleration, the Doppler shift of the reflected wave is negative, the absolute value of the shift increases, and the phone is closer to the ear, allowing for sufficiently strong reflected waves to be received (see figure ⑦ in the figure). During deceleration, the Doppler shift of the reflected wave is negative, the absolute value of the shift decreases, but the phone is farther away from the ear, preventing the reception of a discernible reflected wave (see figure ⑧ in the figure). However, direct waves (see figure ⑥) are always received.
[0082] Finally, the phone is placed far away from the ear again, and can only receive direct waves ⑨, without reflected waves.
[0083] In general, because ③ and ⑧ cannot be actually detected, ②④ are usually called "close features" and ⑥⑦ are called "far features".
[0084] The algorithm determines the screen status based on the received signal. Existing solutions use a binary classification output: whether the phone screen is on (on) or off (off). Off includes the phone being close to the ear (④) and resting at the ear (⑤), while on includes the phone being away from the ear (⑦) and resting at a distance (①⑨). However, in reality, neither the phone being at the ear nor resting at a distance exhibits the Doppler shift characteristic caused by motion, and the magnitude of the amplitude characteristic is unclear. Factors influencing this include multipath effects and the angle between the phone and the ear. This is a drawback of the binary classification approach.
[0085] The time-frequency diagram of the received signal using multiple transmission frequencies is as follows Figure 2 As shown, compared Figure 1 The Doppler frequency characteristics that are more similar to the actual received signal are used (in the shape of an obtuse triangle instead of a slanted line). Figure 2 The three transmit frequencies are equidistant, but in practice, the transmit frequencies can be unequally spaced as long as the minimum spacing is greater than the crosstalk threshold. The crosstalk threshold is the frequency spacing threshold at which the Doppler signatures of adjacent transmit frequencies are mixed into another transmit frequency interval.
[0086] Furthermore, existing solutions treat all frequencies as a whole (i.e., "multi-frequency holistic judgment"). For example, after performing an STFT (Short-Time Fourier Transform), a slice of the time-frequency graph containing all transmitted frequency intervals is cut off and input into the classifier. This has the disadvantage that if an AI (artificial intelligence) model is used as the classifier, the required model size is large. Furthermore, the classifier (including training the AI model and adjusting traditional algorithm parameters) lacks transferability.
[0087] The transferability of a classifier refers to the ability of a classifier trained or parameterized using a dataset with one emission frequency distribution (including number and spacing) to be applied to data from other emission frequency distributions. Multi-frequency holistic judgment has weak transferability, meaning that when training or adjusting the classifier, we can only provide data from a limited number of emission frequency distributions. However, if a new device must use a frequency distribution different from the original data due to factors such as frequency response and interference, the multi-frequency holistic judgment classifier often cannot be used directly. Instead, new data must be collected using the new emission frequency distribution and retraining or parameter adjustment must be performed. This process is time-consuming and labor-intensive, slowing down the application of the algorithm to new devices.
[0088] This embodiment provides an ultrasonic proximity detection method, which is applied to a terminal device. The terminal device includes a speaker and a microphone. The speaker is used to send a transmission signal, and the microphone is used to receive a reception signal corresponding to the transmission signal.
[0089] See also Figure 3 , ultrasonic proximity detection methods include:
[0090] S1. Receive ultrasonic waves.
[0091] Ultrasonic waves are generated by superimposing multiple transmission signals at different frequencies. In this embodiment, the transmission signal is a sine wave, and the spacing between the multiple transmission frequencies can be equal or unequal. As long as the spacing between the transmission frequencies is greater than the crosstalk threshold, the received signals at adjacent frequencies at the same or similar times will not overlap.
[0092] S2. Perform frequency separation on the ultrasonic wave to obtain a receiving signal corresponding to the transmitting signal.
[0093] S3. Input each received signal into the classifier to output the state feature category of the corresponding terminal device.
[0094] Among them, the state feature categories include static features, approaching features and moving away features.
[0095] In an optional embodiment, step S3 includes:
[0096] In response to the frequency of the received signal being greater than the transmit frequency, the output is close to the characteristic.
[0097] In response to the frequency of the received signal being less than the transmit frequency, the output is away from the characteristic.
[0098] In response to the frequency of the received signal being equal to the transmit frequency, a stationary feature is output.
[0099] S4. Summarize the status feature categories corresponding to all received signals, and determine the final status feature of the terminal device according to the summarized status feature categories.
[0100] The final state feature is one of a static feature, an approach feature, and a distance feature.
[0101] This embodiment obtains received signals corresponding to all transmitted signals by frequency separation of received ultrasonic waves, inputs each received signal into a classifier to obtain a state feature category of the terminal device, summarizes and analyzes the state feature categories corresponding to all received signals, and determines the final state feature of the terminal device, thereby improving the ultrasonic proximity detection accuracy of the terminal device and improving the migration capability of the classifier while reducing the computational complexity of the classifier.
[0102] In an optional embodiment, before step S2, the following steps are included:
[0103] S20 , pre-processing the ultrasonic wave to obtain a time-frequency diagram of the ultrasonic wave.
[0104] The preprocessing method includes at least one of short-time Fourier transform, wavelet time-frequency analysis, S transform, and Hilbert-Huang transform.
[0105] Step S2 includes:
[0106] S21. Taking each transmitting frequency as the center, intercept a frequency domain slice of the time-frequency diagram.
[0107] In an optional embodiment, step S2 includes:
[0108] S22. Perform frequency separation on the ultrasonic wave through a peak filter.
[0109] In this embodiment, ultrasound frequency separation can be performed by software processing to extract frequency domain slices of a time-frequency graph centered at each transmission frequency, or directly by hardware using a peak filter. It should be noted that other frequency separation methods known in the art may also be applied to this embodiment.
[0110] In an optional embodiment, step S4 includes:
[0111] S401: In response to the fact that the state feature category of each output corresponding to each received signal is not unique, obtain the confidence level of each output state feature category.
[0112] S402: Perform weighted summation on the confidence level and the frequency of the received signal to obtain a weighted sum value corresponding to each received signal.
[0113] S403: Determine the state feature category corresponding to the first sum value as the final state feature of the terminal device.
[0114] The first sum is the maximum value among the weighted sums.
[0115] In an optional embodiment, step S4 includes:
[0116] S410 : In response to each received signal corresponding to an output state feature category being unique, the number of state feature categories is counted.
[0117] S411: In response to the number of the first feature category being greater than the number of the second feature category, and the number of the first feature category being greater than the number of the third feature category, determining the first feature category as a final state feature of the terminal device.
[0118] The first feature category, the second feature category, and the third feature category are any one of a stationary feature, a close feature, and a far feature.
[0119] S412: In response to the number of approaching features being the same as the number of moving-away features, determining the stationary feature as the final state feature of the terminal device.
[0120] S413: In response to the number of approaching features being the same as the number of stationary features, and the number of moving away features being less than the number of approaching features, determining that the approaching feature is the final state feature of the terminal device.
[0121] S414: In response to the number of the moving-away features being the same as the number of the stationary features, and the number of the approaching features being less than the number of the moving-away features, determining that the moving-away feature is the final state feature of the terminal device.
[0122] In an optional embodiment, step S4 includes:
[0123] Similarly, in response to the fact that the state feature category outputted corresponding to each received signal is unique, the number of the state feature categories is counted.
[0124] S415: In response to the number of moving away features being greater than the number of stationary features, and the number of moving away features being greater than the number of approaching features, determining that the moving away feature is a final state feature of the terminal device.
[0125] S416: In response to the number of approaching features being greater than the number of stationary features, and the number of approaching features being greater than the number of moving away features, determining that the approaching feature is the final state feature of the terminal device.
[0126] S417. On the contrary, that is, the number of far features is less than or equal to the number of static features, or the number of close features is less than or equal to the number of static features, or the number of far features is equal to the number of close features, then the static feature is determined to be the final state feature of the terminal device.
[0127] In an optional embodiment, step S4 includes:
[0128] Similarly, in response to the fact that the state feature category outputted corresponding to each received signal is unique, the number of the state feature categories is counted.
[0129] S418. Obtain current status characteristics of the terminal device.
[0130] Among them, the current status feature is the functional status of the terminal device. For example, for ultrasonic call, the screen is turned off, which is the on or off status of the terminal screen; for ultrasonic gesture page turning, it is the current page number; for ultrasonic gesture volume adjustment, it is the current volume, etc.
[0131] S419: In response to the current state feature being the first state, determining the sum of the number of static features and the old number of close features as the new number of close features.
[0132] S420: In response to the current state feature being in the second state, determining the sum of the number of stationary features and the old number of distant features as the new number of distant features.
[0133] S421: In response to the new number of the approaching feature being greater than the new number of the moving-away feature, determining that the approaching feature is the final state feature of the terminal device.
[0134] S422: In response to the new number of the moving-away feature being greater than the new number of the moving-away feature, determining that the moving-away feature is the final state feature of the terminal device.
[0135] S423: In response to the new number of approaching features being equal to the new number of moving away features, determining that the stationary feature is the final state feature of the terminal device.
[0136] S424. In response to the new number of the approaching feature being equal to the new number of the moving-away feature, and the old number of the approaching feature being equal to the old number of the moving-away feature, determining that the static feature is the final state feature of the terminal device.
[0137] S425. In response to the new number of the approaching feature being equal to the new number of the moving-away feature, and the old number of the approaching feature being greater than the old number of the moving-away feature, determining that the approaching feature is the final state feature of the terminal device.
[0138] S426. In response to the new number of the approaching feature being equal to the new number of the moving-away feature, and the old number of the moving-away feature being greater than the old number of the approaching feature, determining that the moving-away feature is the final state feature of the terminal device.
[0139] In this embodiment, the final state characteristics of the terminal device are determined by a confidence summation method or a voting method, thereby improving the reliability of the final state characteristics. It should be noted that the method for determining the final state characteristics of the terminal device in this embodiment is not limited to the above confidence summation method and voting method.
[0140] In an optional embodiment, the ultrasonic proximity detection method further includes:
[0141] S51. Perform corresponding control on the terminal device according to the final state characteristics.
[0142] Taking the screen-off control during ultrasonic calls as an example, if the final state feature is a proximity feature, the screen of the terminal device is controlled to become a screen-off state; if the final state feature is a distance feature or a stationary feature, the screen of the terminal device is controlled to become a screen-on state.
[0143] In an optional embodiment, the ultrasonic proximity detection method further includes:
[0144] S52: In response to the final state feature of the terminal device appearing a preset number of times in succession, the terminal device is controlled accordingly.
[0145] Taking the screen-off control during ultrasonic calls as an example, if the final state feature is determined to be a proximity feature for more than three consecutive times, the screen of the terminal device is controlled to become a screen-off state; if the final state feature is a distance feature or a stationary feature or cannot be determined to be a proximity feature for more than three consecutive times, the screen of the terminal device is controlled to become a screen-on state.
[0146] In this embodiment, the terminal device is controlled to perform corresponding operations through the final state feature or through the final state feature and the number of times the final state feature appears, thereby improving the accuracy of the control operation of the terminal device.
[0147] The following introduces a specific example to illustrate the ultrasonic proximity detection method of this embodiment in detail.
[0148] Figure 4 This is the flowchart for this example, specifically:
[0149] The speaker of the terminal device emits ultrasonic waves composed of multiple frequency sine waves (transmitting signals). The microphone receives the direct ultrasonic wave and the reflected ultrasonic wave (receiving signal), and converts them into digital signals through the ADC (Analog-to-Digital Converter), i.e., the signal acquisition module 101.
[0150] The preprocessing and frequency division module 102 converts the collected signal into the input for the classifier 103. The most commonly used preprocessing method is STFT, which produces a time-frequency diagram. To facilitate the subsequent "multi-frequency independent judgment and aggregation," frequency separation is also required. For example, frequency domain slices of the time-frequency diagram are taken, centered around each transmit frequency.
[0151] Classifier 103 can use traditional methods (such as calculating the area enclosed by the time-frequency graph features and the direct wave) or AI models (such as CNN, a convolutional neural network, or GRU, a special type of recurrent neural network structure). The output of classifier 103 must be three-category, including stationary features, approaching features, and distant features.
[0152] Taking the screen off during an ultrasonic call as an example, the screen status operations corresponding to the stationary feature, approach feature, and distance feature are hold (hold), out (off), and lit (lit), respectively. Hold corresponds to the state where the phone is relatively stationary with respect to the ear (including the phone being parked next to the ear and at a distance), out corresponds to the phone being close to the ear, and lit corresponds to the phone being far away from the ear. Figure 1 , when detected Figure 1 The close feature ④ in the equation is out, the far feature ⑦ is lit, and no close or far feature is detected, which is hold (①③⑤⑧⑨).
[0153] The classification principles in this example are very clear, and their correspondence with the time-frequency graph features is very clear. In the example of the screen-off ultrasound call, the phone being held to the ear (⑤) and held at a distance (①⑨) are classified into the same category, "hold," effectively reducing the probability of misclassification. This example's classification is based solely on Doppler shift features, without considering amplitude characteristics. This makes the classification criteria very clear and facilitates algorithm judgment. Practical results show that this example can accurately determine the final state characteristics of terminal devices with an accuracy rate of over 95%, while existing two-class classification algorithms only have an accuracy rate of around 70%.
[0154] The current state 104 is the state of the terminal device function module. For example, for ultrasonic call, the screen is turned off, which is the on / off state of the screen; for ultrasonic gesture page turning, it is the current page number; for ultrasonic gesture volume adjustment, it is the current volume.
[0155] The aggregation and post-processing module 105 aggregates the classification results of each frequency from the classifier 103 and combines them with the current state 104 (equivalent to the current screen state hereinafter) to obtain a new state 106 (equivalent to the new screen state hereinafter). The aggregation method may use the information of the current state 104 or may not use the information of the current state 104.
[0156] The following uses the example of turning off the screen during an ultrasonic call to provide an example of the aggregation method. In practice, you can choose one that is applicable, or choose other preset aggregation schemes.
[0157] Example 1: Use the confidence summation method to analyze and summarize the state feature categories:
[0158] The premise of adopting this method is that the output of the classifier 103 is not a single state feature category, but the confidence of each state feature category. When summarizing, the confidence of the state feature category corresponding to the received signal of each frequency is weighted and summed, and then the state feature category with the largest sum is taken as the final classification. It should be noted that the definition of weight is not limited, it can be all 1, can be related to the transmission frequency (the theoretical basis is the Doppler frequency shift formula ), and may also be related to the signal-to-noise ratio (maximum ratio combining), etc.
[0159] Example 2: Using voting method to analyze and summarize status feature categories:
[0160] The premise of adopting this method is that the classifier 103 outputs a single state feature category, and voting is performed when summarizing, and the final result is obtained according to the preset rules. There are three preset rules below, but the actual application is not limited to the following three preset rules.
[0161] Voting rule 1: If a state feature category has the largest number, it is determined as the final state feature. If the number of out and lit is equal (regardless of whether they are greater than / equal to / less than the hold number), the result is hold; if the number of hold and out is tied for more than lit, the result is out; if the number of hold and lit is tied for more than out, the result is lit.
[0162] Voting rule 2: If the number of out is the largest, out is the final state feature; if the number of lit is the largest, lit is the final state feature; in other cases, hold is the final state feature.
[0163] Voting Rule 3: If the current state 104 is off, add the hold count to the out count; if the current state 104 is on, add the hold count to the lit count. Then, directly compare the new out and lit counts; the larger count is the final state feature. If the new out and lit counts are equal, hold is determined to be the final state feature. Alternatively, compare the old out and lit counts; the larger count is determined to be the final state feature. If the old out and lit counts are still equal (indicating that hold count is 0), hold is determined to be the final state feature.
[0164] After summarizing and determining the terminal's final state characteristics, post-processing can be performed according to certain regulations. For example, the screen can be turned off or on using the following table. For example, the screen will be turned off only after the final state characteristics are determined to be out after several consecutive classifications.
[0165]
[0166] The advantages of "multi-frequency independent judgment followed by aggregation" in this example include: a classifier trained or parameter-adjusted on data of one transmission frequency distribution can be directly migrated to data of other transmission frequency distributions without the need to re-collect data, train, or adjust parameters, which can save a lot of time and manpower and enable the algorithm to be quickly applied to new equipment; if an AI model is used as a classifier, because the input of the model is only the information of a single frequency, this method can reduce the size of the model; if an AI model is used as a classifier, this method can also expand the size of the data set. Originally, the multi-frequency information combined was one data, but now the information of each frequency is one data.
[0167] This example implements a single gesture recognition function (for example, only page turning or volume adjustment) through proximity detection based on three state feature categories. Compared to complex gesture recognition, this solution only needs to detect the three state feature categories of stillness, approaching, and moving away, eliminating the need for detailed gesture classification. This simplifies the classifier, reduces the size of the AI model, and reduces the number of parameters.
[0168] Example 2
[0169] Corresponding to the above-mentioned ultrasonic proximity detection method embodiment, the present disclosure also provides an embodiment of an ultrasonic proximity detection system. Figure 5 , the ultrasonic proximity detection system includes:
[0170] The receiving module 1 is used to receive ultrasonic waves, which are generated by superimposing several transmission signals with different transmission frequencies.
[0171] The frequency separation module 2 is used to perform frequency separation on the ultrasonic wave to obtain a receiving signal corresponding to the transmitting signal.
[0172] The feature module 3 is configured to input each received signal into a classifier to output a state feature category of the corresponding terminal device, wherein the state feature category includes a stationary feature, an approaching feature, and a moving feature.
[0173] The determination module 4 is configured to summarize the status feature categories corresponding to all received signals and determine the final status feature of the terminal device according to the summarized status feature categories.
[0174] In an alternative embodiment, see Figure 5 , the ultrasonic proximity detection system also includes:
[0175] The preprocessing module 5 is used to preprocess the ultrasonic wave to obtain a time-frequency diagram of the ultrasonic wave.
[0176] The frequency separation module 2 is further configured to capture frequency domain slices of the time-frequency diagram with each transmitting frequency as the center.
[0177] In an optional embodiment, the frequency separation module 2 is further configured to perform frequency separation on the ultrasonic wave through a peak filter.
[0178] In an alternative embodiment, see Figure 5 , the ultrasonic proximity detection system also includes:
[0179] Acquisition module 6 is used to obtain the confidence of each output state feature category in response to the fact that the state feature category corresponding to each received signal is not unique; it is also used to perform weighted summation of the confidence and the frequency of the received signal to obtain the weighted sum value corresponding to each received signal.
[0180] The determination module 4 is further configured to determine the state feature category corresponding to the first sum value as the final state feature of the terminal device, wherein the first sum value is the maximum value among the weighted sum values.
[0181] In an optional embodiment, the acquisition module 6 is further configured to count the number of status feature categories in response to the fact that the status feature category outputted corresponding to each received signal is unique.
[0182] Determination module 4 is further used to determine that the first feature category is the final state feature of the terminal device in response to the number of the first feature category being greater than the number of the second feature category, and the number of the first feature category being greater than the number of the third feature category; wherein the first feature category, the second feature category and the third feature category are any one of a static feature, a close feature and a distant feature; and is further used to determine that the static feature is the final state feature of the terminal device in response to the number of close features being the same as the number of distant features; and is further used to determine that the close feature is the final state feature of the terminal device in response to the number of close features being the same as the number of static features, and the number of distant features being less than the number of close features; and is further used to determine that the distant feature is the final state feature of the terminal device in response to the number of distant features being the same as the number of static features, and the number of close features being less than the number of distant features.
[0183] In an optional embodiment, the determination module 4 is further used to determine that the far feature is the final state feature of the terminal device in response to the number of far features being greater than the number of stationary features, and the number of far features being greater than the number of close features; it is also used to determine that the close feature is the final state feature of the terminal device in response to the number of close features being greater than the number of stationary features, and the number of close features being greater than the number of far features; it is also used to determine that the stationary feature is the final state feature of the terminal device in response to the above two conditions not being met.
[0184] In an optional implementation, the acquisition module 6 is further configured to acquire current status characteristics of the terminal device.
[0185] The determination module 4 is further configured to, in response to the current state characteristic being the first state, determine the sum of the number of stationary characteristics and the old number of approaching characteristics as the new number of approaching characteristics; and in response to the current state characteristic being the second state, determine the sum of the number of stationary characteristics and the old number of distant characteristics as the new number of distant characteristics; and in response to the new number of approaching characteristics being greater than the new number of distant characteristics, determine the approaching characteristics as the final state characteristic of the terminal device; and in response to the new number of distant characteristics being greater than the new number of approaching characteristics, determine the distant characteristics as the final state characteristic of the terminal device; and in response to the new number of approaching characteristics being equal to the new number of distant characteristics, determine the distant characteristics as the final state characteristic of the terminal device. quantity, then the static feature is determined to be the final state feature of the terminal device; it is also used to determine that the static feature is the final state feature of the terminal device in response to the new quantity of the approaching feature being equal to the new quantity of the moving-away feature, and the old quantity of the approaching feature being equal to the old quantity of the moving-away feature; it is also used to determine that the approaching feature is the final state feature of the terminal device in response to the new quantity of the approaching feature being equal to the new quantity of the moving-away feature, and the old quantity of the approaching feature being greater than the old quantity of the moving-away feature; it is also used to determine that the moving-away feature is the final state feature of the terminal device in response to the new quantity of the approaching feature being equal to the new quantity of the moving-away feature, and the old quantity of the moving-away feature being greater than the old quantity of the approaching feature.
[0186] In an alternative embodiment, see Figure 5 , the ultrasonic proximity detection system also includes:
[0187] The control module 7 is used to control the terminal device accordingly according to the final state characteristics.
[0188] In an optional embodiment, the control module 7 is further configured to perform corresponding control on the terminal device in response to the final state feature of the terminal device appearing continuously for a preset number of times.
[0189] In an optional embodiment, the feature module 3 is further used to output a close feature in response to the frequency of the received signal being greater than the transmitting frequency; is also used to output a far feature in response to the frequency of the received signal being less than the transmitting frequency; and is also used to output a stationary feature in response to the frequency of the received signal being equal to the transmitting frequency.
[0190] Since the system embodiments generally correspond to the method embodiments, reference will be made to the description of the method embodiments for relevant details. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components of the units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the disclosed solution.
[0191] Example 3
[0192] Figure 6 This is a structural diagram of an electronic device showing an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and configured to run on the processor. When the processor executes the computer program, the ultrasonic proximity detection method of any of the above embodiments is implemented. Figure 6 The electronic device 60 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.
[0193] like Figure 6 As shown, the electronic device 60 may be a general-purpose computing device, such as a server device. Components of the electronic device 60 may include, but are not limited to, the at least one processor 61, the at least one memory 62, and a bus 63 connecting different system components (including the memory 62 and the processor 61).
[0194] The bus 63 includes a data bus, an address bus, and a control bus.
[0195] The memory 62 may include a volatile memory, such as a random access memory (RAM) 621 and / or a cache memory 622 , and may further include a read-only memory (ROM) 623 .
[0196] The memory 62 may also include a program tool 625 (or utility) having a set (at least one) of program modules 624, such program modules 624 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.
[0197] The processor 61 executes various functional applications and data processing by running the computer programs stored in the memory 62 , such as the ultrasonic proximity detection method provided in any of the above embodiments.
[0198] The electronic device 60 can also communicate with one or more external devices 64 (e.g., a keyboard, pointing device, etc.). Such communication can occur via an input / output (I / O) interface 65. Furthermore, the electronic device 60 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 66. As shown, the network adapter 66 communicates with other modules of the electronic device 60 via a bus 63. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the electronic device 60, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.
[0199] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.
[0200] Example 4
[0201] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the ultrasonic proximity detection method provided by any of the above embodiments is implemented.
[0202] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.
[0203] Example 5
[0204] An embodiment of the present disclosure further provides a computer program product, including a computer program, which implements any of the above-mentioned ultrasonic proximity detection methods when executed by a processor.
[0205] The program code for executing the computer program product of the present disclosure may be written in any combination of one or more programming languages, and the program code may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.
[0206] While specific embodiments of the present disclosure have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of protection of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, and such changes and modifications are intended to fall within the scope of protection of the present disclosure.
Claims
1. An ultrasonic proximity detection method, characterized in that: Applied to terminal equipment, the ultrasonic proximity detection method includes: Receiving ultrasonic waves; wherein the ultrasonic waves are formed by superimposing a plurality of transmission signals with different transmission frequencies; Performing frequency separation on the ultrasonic wave to obtain a receiving signal corresponding to the transmitting signal; Input each received signal into a classifier to output a corresponding state feature category of the terminal device; wherein the state feature category includes a stationary feature, a close feature, and a far feature; Summarize the status feature categories corresponding to all received signals, and determine the final status feature of the terminal device according to the summarized status feature categories.
2. The ultrasonic proximity detection method according to claim 1, characterized in that: The step of performing frequency separation on the ultrasonic wave comprises: Preprocessing the ultrasonic wave to obtain a time-frequency diagram of the ultrasonic wave; The step of performing frequency separation on the ultrasonic wave comprises: A frequency domain slice of the time-frequency diagram is cut out with each transmission frequency as the center.
3. The ultrasonic proximity detection method according to claim 1, characterized in that: The step of performing frequency separation on the ultrasonic wave comprises: The ultrasonic waves are subjected to frequency separation by a peak filter.
4. The ultrasonic proximity detection method according to claim 1, characterized in that: The step of determining the final state feature of the terminal device according to the summarized state feature category comprises: In response to the state feature category of each received signal corresponding to the output being not unique, obtaining the confidence of the state feature category of each output; Performing weighted summation of the confidence level and the frequency of the received signal to obtain a weighted summation value corresponding to each received signal; Determine the state feature category corresponding to the first sum value as the final state feature of the terminal device; Wherein, the first sum value is the maximum value among the weighted sum values; or, In response to each received signal corresponding to the output state feature category being unique, counting the number of the state feature categories; In response to the number of the first feature category being greater than the number of the second feature category, and the number of the first feature category being greater than the number of the third feature category, determining that the first feature category is a final state feature of the terminal device; wherein the first feature category, the second feature category, and the third feature category are any one of a stationary feature, a close feature, and a far feature; In response to the number of approaching features being the same as the number of moving away features, determining the stationary feature as a final state feature of the terminal device; In response to the number of approaching features being the same as the number of stationary features, and the number of moving away features being less than the number of approaching features, determining the approaching feature as a final state feature of the terminal device; In response to the number of the moving-away features being the same as the number of the stationary features, and the number of the approaching features being less than the number of the moving-away features, determining that the moving-away features are the final state features of the terminal device; or, In response to the number of moving away features being greater than the number of stationary features, and the number of moving away features being greater than the number of approaching features, determining that the moving away feature is a final state feature of the terminal device; In response to the number of approaching features being greater than the number of stationary features, and the number of approaching features being greater than the number of moving away features, determining the approaching feature as a final state feature of the terminal device; Otherwise, the static feature is determined to be the final state feature of the terminal device; or, Obtaining current status characteristics of the terminal device; In response to the current state feature being the first state, determining the sum of the number of stationary features and the old number of close features as the new number of close features; In response to the current state feature being the second state, determining the sum of the number of stationary features and the old number of far-away features as the new number of far-away features; In response to the new number of the approaching feature being greater than the new number of the moving away feature, determining the approaching feature as a final state feature of the terminal device; In response to the new number of the moving away feature being greater than the new number of the moving towards feature, determining that the moving away feature is a final state feature of the terminal device; In response to the new number of the approaching feature being equal to the new number of the moving away feature, determining the stationary feature as a final state feature of the terminal device; In response to the new number of the approaching feature being equal to the new number of the moving away feature, and the old number of the approaching feature being equal to the old number of the moving away feature, determining that the stationary feature is a final state feature of the terminal device; In response to the new number of the approaching feature being equal to the new number of the moving away feature, and the old number of the approaching feature being greater than the old number of the moving away feature, determining that the approaching feature is a final state feature of the terminal device; In response to the new number of the approaching feature being equal to the new number of the moving away feature, and the old number of the moving away feature being greater than the old number of the approaching feature, it is determined that the moving away feature is the final state feature of the terminal device.
5. The ultrasonic proximity detection method according to claim 1, characterized in that: The ultrasonic proximity detection method further comprises: Controlling the terminal device accordingly according to the final state characteristics; or, In response to the final state feature of the terminal device appearing continuously for a preset number of times, the terminal device is controlled accordingly.
6. The ultrasonic proximity detection method according to claim 1, characterized in that: The step of inputting each received signal into a classifier respectively to output a corresponding state feature category of the terminal device; wherein the state feature category includes a stationary feature, a close feature, and a far feature comprises: In response to the frequency of the received signal being greater than the transmit frequency, outputting a proximity feature; In response to the frequency of the received signal being less than the transmit frequency, outputting a distance characteristic; In response to the frequency of the received signal being equal to the transmit frequency, a stationary feature is output.
7. An ultrasonic proximity detection system, characterized in that: Applied to terminal equipment, the ultrasonic proximity detection system comprises: A receiving module, used for receiving ultrasonic waves; wherein the ultrasonic waves are formed by superposition of a plurality of transmission signals with different transmission frequencies; A frequency separation module, used for performing frequency separation on the ultrasonic wave to obtain a receiving signal corresponding to the transmitting signal; A feature module, used to input each received signal into a classifier to output a corresponding state feature category of the terminal device; wherein the state feature category includes a stationary feature, a close feature, and a far feature; The determination module is used to summarize the status feature categories corresponding to all received signals, and determine the final status feature of the terminal device according to the summarized status feature categories.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the ultrasonic proximity detection method according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the ultrasonic proximity detection method according to any one of claims 1 to 6 is implemented.
10. A chip, applied to electronic equipment, characterized in that: The chip is used to execute the ultrasonic proximity detection method according to any one of claims 1 to 6.
11. A chip module, applied to electronic equipment, characterized in that: The invention comprises a transceiver component and a chip, wherein the chip is used to execute the ultrasonic proximity detection method according to any one of claims 1 to 6.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the ultrasonic proximity detection method according to any one of claims 1 to 6 is implemented.
