Processing method and electronic equipment

Abstract

The invention provides a processing method and electronic equipment, and is applied to the technical field of electronics. The method is applied to first electronic equipment and comprises the steps that in response to first operation, first data are obtained through a first detection assembly of the first electronic equipment, second data are obtained through a second detection assembly of the first electronic equipment, and the first detection assembly and the second detection assembly are located at different positions of the first electronic equipment; the detection assemblies are detection assemblies with different functions; according to the first data and the second data, whether the first operation is a target operation is determined, and the target operation is used for controlling the first electronic equipment to execute a function corresponding to the target operation.

Description

Technical Field

[0001] This disclosure relates to the field of electronic technology, and more particularly to a processing method and an electronic device. Background Technology

[0002] With the advancement of electronic device performance, human-computer interaction methods are becoming increasingly diversified. However, the current reliability of human-computer interaction operation recognition is relatively poor, affecting the effectiveness of human-computer interaction. Summary of the Invention

[0003] In view of this, the present disclosure provides a processing method and an electronic device.

[0004] According to a first aspect of this disclosure, a processing method is provided, applied to a first electronic device, the method comprising: in response to a first operation, obtaining first data through a first detection component of the first electronic device, obtaining second data through a second detection component of the first electronic device, wherein the first detection component and the second detection component are located at different locations in the first electronic device and are detection components with different functions; determining, based on the first data and the second data, whether the first operation is a target operation, wherein the target operation is used to control the first electronic device to perform a function corresponding to the target operation.

[0005] According to an embodiment of this disclosure, in the method of claim 1, the first detection component is used to collect vibration information generated by the first electronic device due to the first operation, and the second detection component is used to collect acoustic information generated by the first operation.

[0006] According to embodiments of this disclosure, in response to a first operation, obtaining first data through a first detection component of a first electronic device and obtaining second data through a second detection component of the first electronic device includes: in response to the first operation, obtaining first data through the first detection component of the first electronic device; and if the first data satisfies a target condition, obtaining second data through the second detection component of the first electronic device, wherein the target condition characterizes vibration information of the first electronic device in a vibration mode.

[0007] According to embodiments of this disclosure, the method further includes: obtaining scene information of a first electronic device, the scene information including interconnection information and / or operation information of the first electronic device; and determining, based on the scene information, a target operation triggers a target interactive function of the first electronic device.

[0008] According to embodiments of this disclosure, the method further includes: determining a target location based on first data and second data, wherein the target location represents location information of the first operation acting on the first electronic device; and determining a target interactive function of the first electronic device triggered by the target operation based on the target location.

[0009] According to embodiments of this disclosure, the method further includes: if the first operation is a target operation and the second electronic device is in an idle state, switching the third electronic device currently wirelessly connected to the first electronic device to the second electronic device, wherein the historical connection frequency between the second electronic device and the first electronic device is greater than the historical connection frequency between the third device and the first electronic device.

[0010] According to embodiments of this disclosure, the method further includes: if the first operation is a target operation and / or the first operation acts on a target location of the first electronic device, activating the image acquisition device of the first electronic device, and determining the interactive function of the image acquisition device based on the state and / or operating information of the first electronic device.

[0011] According to embodiments of this disclosure, determining whether a first operation is a target operation based on first data and second data includes: inputting the first data and second data into a first model to obtain a first output result, wherein the first output result indicates that the first operation is a target operation; or inputting the first data and second data into the first model to obtain a second output result, wherein the second output result indicates that the first operation is a non-target operation.

[0012] According to embodiments of this disclosure, the first model is obtained through the following operations: acquiring multiple sample data, including positive samples and negative samples, where the positive samples are third and fourth data corresponding to the target operation, the third data being information about the target operation acting on the first electronic device causing vibration, and the fourth data being acoustic information generated by the target operation acting on the first electronic device; and the negative samples are fifth and sixth data corresponding to non-target operations, where the fifth data is information about the non-target operation acting on the first electronic device causing vibration, and the sixth data is acoustic information generated by the non-target operation acting on the first electronic device. The target operation and non-target operation are the true labels of the sample data; inputting the sample data into the first model to obtain prediction results, where the prediction results are operation labels corresponding to the sample data, and the operation labels include the target operation and non-target operation; updating the parameters of the first model based on the difference between the prediction results and the true labels until the first model converges, thus obtaining the trained first model.

[0013] According to an embodiment of this disclosure, determining whether a first operation is a target operation based on first data and second data includes: acquiring third data of a second electronic device, wherein the third data is vibration information generated by the second electronic device due to the first operation acting on it; and determining whether the first operation is a target operation based on the first data, the second data, and the third data.

[0014] A second aspect of this disclosure provides a first electronic device, comprising: at least one first detection component for acquiring first data of the first electronic device in response to a first operation; at least one second detection component for acquiring second data of the first electronic device in response to the first operation; a processor for acquiring the first data through the first detection component and acquiring the second data through the second detection component of the first electronic device in response to the first operation, wherein the first detection component and the second detection component are located at different locations in the first electronic device and are detection components with different functions; and determining whether the first operation is a target operation based on the first data and the second data, wherein the target operation is used to control the first electronic device to perform a function corresponding to the target operation.

[0015] A third aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0016] A fourth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 This diagram illustrates an application scenario of the processing method according to an embodiment of the present disclosure.

[0020] Figure 2 A flowchart illustrating a processing method according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 3 A schematic diagram of a first electronic device according to an embodiment of the present disclosure is shown;

[0022] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a processing method according to an embodiment of the present disclosure. Detailed Implementation

[0023] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0027] This disclosure provides a processing method and an electronic device. Before introducing the technical solutions provided by this disclosure, the relevant technologies involved in this disclosure will be described first.

[0028] With the advancement of electronic device performance, human-computer interaction methods are becoming increasingly diversified. However, the current reliability of human-computer interaction operation recognition is relatively poor, affecting the effectiveness of human-computer interaction.

[0029] In one example, electronic devices commonly use "tap" to complete interactive functions such as Bluetooth pairing and near-field sharing. In recent years, this interaction has been gradually introduced into laptops, for example, through NFC near-field communication to sense device contact. This method detects the contact point via antenna coupling and can reliably trigger the connection at close range in a fixed location. However, it requires additional NFC antennas, controllers, and a window design in the casing, resulting in higher hardware costs and the inability to provide collision direction information. Therefore, the solutions in this area have shortcomings in terms of reliability, cost, and functional completeness. Furthermore, a single sensing mode cannot simultaneously achieve high reliability, low power consumption, and functional scalability.

[0030] Figure 1 The diagram illustrates an application scenario of the processing method according to an embodiment of the present disclosure.

[0031] like Figure 1 As shown, the application scenario includes a first electronic device 100 and a second electronic device 200. In this embodiment, the first electronic device 100 integrates a first detection component and a second detection component.

[0032] For example, if a user bumps a second electronic device 200 into a first electronic device 100 (first operation), the first electronic device 100 responds to the collision operation by obtaining first data through a first detection component and second data through a second detection component. The first and second detection components are located at different positions on the first electronic device and are detection components with different functions. Based on the first and second data, it is determined whether the first operation is a collision operation. The target operation is used to control the first electronic device to perform the function corresponding to the collision operation. Examples include screen mirroring, application relay, content sharing, and device sharing.

[0033] In this disclosure, the first electronic device 100 may be, for example, a mobile phone, tablet computer, personal computer (PC), personal digital assistant (PDA), netbook, wearable electronic device (e.g., smartwatch, smart bracelet, augmented reality (AR) device, virtual reality (VR) device, etc.), in-vehicle device, smart speaker, and smart home device (e.g., smart refrigerator, smart TV, smart air conditioner), etc. This application does not impose any special restrictions on the specific form of the first electronic device 100.

[0034] The following will be based on Figure 1 The described scene, through Figure 2 The processing method of the embodiments of this disclosure will be described in detail.

[0035] Figure 2 A flowchart illustrating a processing method according to an embodiment of the present disclosure is shown schematically.

[0036] like Figure 2 As shown, the processing method of this embodiment is applied to a first electronic device, and the method includes operations S210 to S220.

[0037] The first electronic device includes, but is not limited to: laptops, tablets, all-in-one computers, smart displays, smart home control screens, and other devices with computing capabilities and human-computer interaction requirements.

[0038] In operation S210, in response to the first operation, first data is obtained through the first detection component of the first electronic device, and second data is obtained through the second detection component of the first electronic device. The first detection component and the second detection component are located at different positions in the first electronic device and are detection components with different functions.

[0039] In operation S220, based on the first data and the second data, it is determined whether the first operation is a target operation. The target operation is used to control the first electronic device to perform the function corresponding to the target operation.

[0040] For example, the first operation may be a user making a short mechanical impact on the body of the first electronic device with a part of their body, such as the user tapping the palm rest area of ​​a laptop with their finger; the first operation may also be a user making a short mechanical impact on the body of the first electronic device while holding a second electronic device, such as the user touching the bezel of a laptop screen with their mobile phone.

[0041] The first detection component and the second detection component can be sensing devices based on different physical principles within the first electronic device. For example, the first detection component can be an accelerometer, a vibration sensor, etc.; the second detection component can be a microphone, a Hall sensor, etc. The first detection component and the second detection component are located in different positions within the first electronic device, and their corresponding functions within the first electronic device are different. For example, the first detection component can be an accelerometer located near the hinge of a laptop; the second detection component can be a microphone located near the camera module above the keyboard of a laptop.

[0042] The first data can be information of a first dimension generated by the first detection component during the first operation applied to the first electronic device. For example, the first data can be vibration information, deformation information, etc. For instance, if a user taps the palm rest area of ​​a laptop with their finger, and the first detection component is an accelerometer, the first data can be vibration information collected by the accelerometer regarding the vibration generated in the laptop caused by the finger tapping the palm rest area.

[0043] The second data can be information generated by the second detection component in a second dimension during the first operation applied to the first electronic device. For example, the second data can be acoustic information, thermal information, etc. For instance, if a user taps the palm rest area of ​​a laptop with their finger, and the second detection component is a microphone, the second data can be acoustic information collected by the microphone regarding the impact of the finger tapping the palm rest area on the laptop, causing the laptop to generate acoustic information.

[0044] A target operation can be a physical interaction action or a pre-defined action applied by a user to a first electronic device, intended to trigger the device to perform a specific function. For example, a target operation could be a user briefly tapping the first electronic device at a relatively perpendicular angle with their finger or another device to trigger a first function; for instance, a user tapping the palm rest area of ​​a laptop to turn on the laptop's camera. A target operation could also be a combination of two or more taps performed by the user within a short period (e.g., within 500ms) to trigger a second function; for example, a user tapping the palm rest area of ​​a laptop twice consecutively can switch the Bluetooth device connected to the laptop. A target operation could also be a brief swipe of the user's finger or object on the device's surface to trigger a third function; for example, a user briefly swiping their finger across the laptop's screen can take a screenshot. A target operation could also be smoothly placing another device against a specific area of ​​this device (a tap) to trigger a third function; for example, a user placing their phone near the laptop's camera can transfer files.

[0045] It should be noted that the embodiments disclosed herein do not specifically limit the function corresponding to the target operation, but can be associated with the function of the first electronic device regarding the target operation in actual application.

[0046] It is understandable that by using detection components with different functions in electronic devices to collect data on the first electronic device caused by the first operation from different dimensions, and by analyzing and judging whether the first operation is the target operation, it is possible not only to improve the accuracy of target operation detection, but also to improve the reliability of interactive recognition in complex usage environments.

[0047] As described above, in some embodiments, the first detection component is used to collect vibration information generated by the first electronic device due to the first operation, and the second detection component is used to collect acoustic information generated by the first operation.

[0048] For example, the first detection component can be a sensor based on the principle of inertial measurement, used to sense the mechanical vibration caused by the first operation. For instance, the first detection component can be an accelerometer (G-sensor), a gyroscope, a vibration sensor, etc. When the first operation is applied to the first electronic device, the mechanical impact will generate structurally propagating vibrations such as bending waves and shear waves in the device's casing or structural components. The first detection component can capture characteristics such as the amplitude, frequency, and duration of these vibrations. For example, when a user taps the palm rest area of ​​a laptop with their finger, an accelerometer located on the laptop motherboard or near the hinge can detect the triaxial acceleration change generated at the moment of the tap; this change curve is part of the vibration information.

[0049] The second detection component can be a sensor based on the principle of acoustic perception, used to capture the acoustic wave signal triggered by the first operation. For example, the second detection component can be a microphone, an ultrasonic sensor, etc. When the first operation is applied to the first electronic device, the mechanical impact not only generates structural vibration, but also excites sound waves through the coupling between the device surface and the air, or directly generates impact sound in the air. The second detection component can collect information such as the time-domain waveform, spectral characteristics, and energy distribution of these sound waves. For example, when a user touches the bezel of a laptop screen with their mobile phone, a microphone located near the camera module above the laptop keyboard can record the short sound generated at the moment of touch; the audio signal of this sound is the acoustic information.

[0050] It should be noted that vibration and acoustic information are two different physical responses triggered by the same first operation. Vibration information mainly propagates through solid media (such as metal frames, plastic shells, circuit boards, etc.), with a fast propagation speed and relatively slow attenuation, and is sensitive to the onset time and location of the mechanical impact. Acoustic information mainly propagates through air, with a relatively slow propagation speed and a wider frequency band, and can reflect the intensity of the impact and the acoustic characteristics of the environment. The two types of information are highly correlated in the time domain. That is, when the first operation occurs, the first detection component and the second detection component almost simultaneously (usually within a time window of 1 to 3 milliseconds) capture the onset of the vibration signal and the onset of the acoustic signal, respectively. This temporal consistency provides a reliable physical basis for subsequent judgment on whether the first operation is the target operation.

[0051] In one example, a user wants to quickly activate the camera function while using a laptop. The user taps the palm rest area on the right side of the laptop's C-side (keyboard side) with their finger. This tap causes a momentary deformation of the laptop's casing. The mechanical waves generated by this deformation propagate inward along the metal support structure of the palm rest. An accelerometer located on the motherboard near the touchpad module detects a sharp acceleration peak in the Z-axis direction (perpendicular to the keyboard plane), lasting approximately 5 to 10 milliseconds, which then rapidly decays. This peak and its time-domain waveform constitute vibration information. Simultaneously, the impact of the finger on the palm rest surface generates a short impact sound near the contact point. This sound wave propagates through the air, and a microphone located next to the webcam at the top center of the laptop screen captures an audio pulse lasting approximately 10 to 20 milliseconds. The main energy of this pulse is concentrated in the 2 to 8 kHz frequency band. The time-domain waveform and spectral characteristics of this audio pulse constitute acoustic information. After the laptop's processor reads the vibration information output by the accelerometer and the acoustic information output by the microphone, it compares and analyzes the two data and finds that they are highly synchronized in time and have the same energy change trend. Therefore, it determines that the tapping action is the user's preset target operation, and then triggers the camera to start up quickly and enter standby mode.

[0052] Understandably, the dual-modal sensing method, which uses the first detection component to collect vibration information and the second detection component to collect acoustic information, can cross-validate the first operation from different physical dimensions. This not only effectively reduces false triggering caused by environmental noise or the device's own movement, but also improves the accuracy and robustness of target operation recognition through the consistency between the two types of information, enabling the first electronic device to reliably respond to the user's interactive intent even in complex usage scenarios.

[0053] As described above, in operation S210, in response to the first operation, first data is obtained through the first detection component of the first electronic device, and second data is obtained through the second detection component of the first electronic device. In one possible implementation, the operation may further include the operation of: in response to the first operation, obtaining the first data through the first detection component of the first electronic device; and if the first data satisfies a target condition, obtaining the second data through the second detection component of the first electronic device, wherein the target condition characterizes vibration information of the first electronic device in a vibration mode.

[0054] For example, in this implementation, the first electronic device employs a hierarchical triggering mechanism to control the operating states of the first and second detection components, thereby achieving efficient sensing under low power consumption. Specifically, the first electronic device normally keeps the first detection component in a low-power protection mode, continuously monitoring whether the device vibrates; while the second detection component is normally in standby or off state, and is only activated and begins data acquisition after the first detection component detects vibration information that meets specific conditions. This hierarchical triggering method can avoid the power consumption overhead caused by the second detection component operating at a high sampling rate for a long time, while ensuring rapid acquisition of multimodal data when needed.

[0055] The target conditions can be a set of preset thresholds or characteristic patterns used to determine whether vibration information may be caused by a user's intentional physical interaction. For example, target conditions may include one or more combinations of characteristics such as vibration amplitude exceeding a preset threshold, vibration duration within a preset range, and vibration waveform having a sharp rising edge. For instance, when the first detection component is an accelerometer, the target conditions can be set as follows: the peak acceleration detected in any single-axis direction is greater than 0.5g, where g is the acceleration due to gravity, and the duration of the vibration pulse corresponding to this peak is within the range of 3 to 30 milliseconds, while the slope of the pulse rising edge is greater than a certain preset value. The purpose of setting these conditions is to initially screen out vibration events that may be caused by target operations such as tapping or touching, while excluding non-target vibrations such as natural placement of equipment, slight table vibration, or fan operation.

[0056] Once the first detection component acquires the first data, the processor or embedded controller of the first electronic device immediately performs a rapid analysis to determine if it meets the target conditions. If the first data meets the target conditions, the processor sends an activation command to the second detection component within a very short time (usually within 1 to 5 milliseconds), causing the second detection component to switch from standby mode to operating mode and begin acquiring acoustic information. If the first data does not meet the target conditions, such as if the vibration amplitude is too small, the duration is too long, or the waveform does not conform to the expected characteristics, the processor determines that the vibration was not caused by the user's target operation, and the second detection component remains in standby mode and is not activated, thereby avoiding unnecessary data acquisition and processing overhead.

[0057] It should be noted that after the second detection component is activated, it will continuously collect acoustic information within a preset time window. The length of this time window is usually set to 10 to 20 milliseconds to ensure that the acoustic response corresponding to the first data can be fully captured. After the time window ends, if no new vibration information that meets the target conditions is detected, the second detection component will automatically return to standby mode, thereby minimizing the overall power consumption of the device.

[0058] In one example, a user wants to trigger a Bluetooth device switching function by tapping the palm rest area while using a laptop. Under normal conditions, the accelerometer on the motherboard operates continuously at a low sampling rate (e.g., 100 Hz) to monitor the device's vibration in real time. The microphone near the camera module is in standby mode and not sampling audio, keeping the power consumption of the entire sensing system low (approximately 0.5 to 1 milliwatt). When the user taps the right palm rest area of ​​the laptop quickly with their finger, the tap causes a momentary deformation of the palm rest's metal support plate. The resulting mechanical wave propagates towards the motherboard, and the accelerometer detects a sharp peak in Z-axis acceleration with an amplitude of 1.2g, a duration of approximately 8 milliseconds, and a rise-edge slope of 0.6g per millisecond. Upon receiving the first data output from the accelerometer, the laptop's embedded controller (EC) immediately compares this data with preset target conditions: the peak value of 1.2g is greater than the threshold of 0.5g, the duration of 8 milliseconds is within the range of 3 to 30 milliseconds, and the rise-edge slope meets the characteristics of a rapid impact. Therefore, the EC determines that the first data meets the target conditions. Within 2 milliseconds of the judgment being completed, the EC sends an activation command to the microphone, which switches from standby mode to working mode and increases the sampling rate to 48 kHz to begin collecting acoustic information. After comprehensively analyzing the first data (vibration information) and the second data (acoustic information), the EC confirms that the tapping action is the user's preset target operation, thereby triggering the Bluetooth device switching process and switching the audio output from the currently connected Bluetooth headset A to the Bluetooth speaker B.

[0059] Understandably, by first determining whether the vibration information meets the target conditions through the first detection component, and then activating the second detection component to collect acoustic information as needed, the hierarchical triggering mechanism can not only significantly reduce the power consumption of the first electronic device in standby or daily use scenarios, but also reduce the computational burden of the processor in processing invalid data while ensuring the accuracy of target operation recognition, thereby improving the overall energy efficiency and response speed of the system.

[0060] In some embodiments, determining whether a first operation is a target operation based on first data and second data includes: determining a first feature based on the first data, the first feature including a first time feature and a first energy feature, the first time feature being the moment when the first peak in the structure wave occurs, the structure wave being a curve showing the change in vibration intensity of the casing of the first electronic device in the target direction over time, and the first energy feature being the maximum peak value of the structure wave; determining a second feature based on the second data, the second feature including a second time feature and a second energy feature, the second time feature being the start time of the sound wave, and the second energy feature being the total energy of the sound wave; and determining a target evaluation value based on the first feature and the second feature, the target evaluation value characterizing the degree of matching between the first operation and the collision operation.

[0061] In one example, a collision would generate both structurally propagating vibrational signals on the laptop casing and airborne acoustic signals.

[0062] • The inertial mode (G-sensor) captures the bending / shear waveforms of the housing panels, which have an extremely fast response and are sensitive to the initial transient.

[0063] • Acoustic mode (Mic) captures the radiated sound of impact sound coupled with structure-air, which has a wider frequency band and is sensitive to environmental noise.

[0064] Both are triggered by the same event (collision), and their start times are almost synchronized (common delays). Therefore, it can be used for consistency verification and complementary positioning.

[0065] Suppose a user taps the palm rest area of ​​a laptop with their finger; record the location where the tapping action occurs. This triggers the laptop to generate structural waves. With sound waves The structure wave is acquired through the laptop's G-sensor, and the sound wave is acquired through the laptop's microphone. In the... accelerometer and the first Observations were made at each microphone:

[0066]

[0067] in, For the transfer function related to location and structure / acoustic path, This is noise. The start time, energy distribution, and rise edge shape of the collision operation correspond to each other in the two modes.

[0068] Extracting the first temporal feature from the first data: the time of the first peak in the structural wave. First energy meter characteristics: the maximum peak value or short-time energy of the structural wave. In other embodiments, the first feature may also include a first sub-feature: the slope of the rising edge of the structural wave (the average slope from the trigger point to the peak point) or a second sub-feature: when multiple G-sensors are configured, the arrival time of the impact signal received by each sensor can be calculated, and then the time difference between each pair can be calculated to estimate the collision location and provide regional discrimination capability.

[0069] Extract the second time and feature from the second data: the start time of the sound wave. The total energy of sound waves (full band or sub-band energy of each frequency band). In other embodiments, the second feature may further include a second sub-feature: the spectral kurtosis or zero crossover rate of the sound wave, and the time delay of the multiple microphones with respect to the sound wave signal, for estimating the azimuth angle of the sound source and providing localization.

[0070] Based on the first and second features, the degree of matching between the user's typing action and the actual typing action is further calculated (target evaluation value). These features are all calculated within a synchronization window of 10–15 ms to ensure temporal alignment. If the target evaluation value is greater than the target threshold (e.g., 98%), it indicates that the user's typing action is a real typing action, triggering the target function set by the laptop regarding the typing action (e.g., waking up the camera).

[0071] In some embodiments, determining a target evaluation value based on a first feature and a second feature includes: determining a first similarity based on the first feature, wherein the first similarity characterizes the similarity between deformation information and the deformation of a collision operation; determining a second similarity based on the second feature, wherein the second similarity characterizes the acoustic similarity between acoustic information and the acoustics of a collision operation; determining a target confidence level based on the first feature and the second feature, wherein the target confidence level characterizes the degree of confidence that the deformation information and the acoustic information are for the same collision operation; and determining a target evaluation value based on the first similarity, the second similarity, and the target confidence level.

[0072] In one example, the log-likelihood ratio to the first similarity is calculated for both the G-sensor modality and the microphone modality. Second similarity For the G-sensor mode, the log-likelihood ratio is defined as: .in, This represents the relevant features extracted by the G-sensor. This indicates the assumption that a target operation exists. This represents the assumption that the target operation (i.e., interference or noise) does not exist. Probability density function. This can be obtained through offline training: collecting a large number of labeled samples (including real collisions and various disturbances), and statistically modeling the feature distribution of each class of samples (such as Gaussian mixture models or kernel density estimation). The log-likelihood ratio of the microphone modalities... Similarly.

[0073] Finally, the log-likelihood ratios of the two modalities (first similarity and second similarity) are weighted and fused with the target confidence:

[0074]

[0075] in, The weighting is determined by the fusion parameters. Because the G-sensor is more sensitive to transient collision responses and has stronger resistance to environmental noise, this embodiment assigns it a larger weight. The microphone provides frequency domain details and directional information, weighted... ;Target confidence level serves as a reliability guarantee, weight .if If the first operation is determined to be the target operation, that is, the user's tapping operation is a real tapping operation; if If the first operation is not the target operation, then it is determined that the user's tapping operation is not a real tapping operation; if If the result is uncertain, you can choose to refuse or ask the user for secondary confirmation.

[0076] In some embodiments, determining a target confidence level based on a first feature and a second feature includes: determining a first confidence level based on a time difference and a target time range, wherein the time difference is the difference between the first time feature and the second time feature, the first confidence level characterizes the degree of confidence that the first operation is a collision operation in the time dimension, and the target time range is the range of values ​​for the time deviation between the first time feature and the second time feature in the collision operation; determining a second confidence level based on an energy ratio and a target energy range, wherein the energy ratio is the ratio of the first energy feature and the second energy feature, the second confidence level is the degree of confidence that the first operation is a touch operation in the energy dimension, and the target energy range is the range of values ​​for the energy deviation between the first energy feature and the second energy feature in the collision operation; and determining a target confidence level based on the first confidence level and the second confidence level.

[0077] In one example, the time difference is calculated based on the first time feature and the second time feature. . This represents the start time of the microphone signal. This represents the peak time of the G-sensor signal. If... Here, 1–3ms is the target time range, indicating good temporal consistency, thus assigning a first confidence level. ;like If the temporal consistency is weak, then a first confidence level is assigned. ;like or

[0078] If the time domains are inconsistent, then the first confidence level is assigned. .

[0079] Calculate the energy ratio based on the third and fourth time characteristics. . The total energy of the sound wave. This represents the total energy of the structure wave. The ratio is set to a very small positive number (e.g., 0.01) to prevent the denominator from being zero. This ratio reflects the energy coupling relationship between the two modal signals. Theoretically, for the same collision event, the greater the impact intensity, the greater the vibration energy measured by the G-sensor, and the greater the radiated acoustic energy; therefore, the two should be positively correlated. However, since structural propagation efficiency and acoustic radiation efficiency are affected by various factors such as materials, geometry, and boundary conditions, this correlation is not strictly linear. Based on experimental data statistics, for real collision events, if... If the energy consistency is good, then a second confidence level is assigned. ;if If the energy consistency is weak, a second confidence level is assigned. ;if If the energy is inconsistent, a second confidence level is assigned. The target confidence level is obtained by weighting and fusing the first and second confidence levels. . Where a and b are weighting coefficients, such as a=0.6 and b=0.4.

[0080] As described above, the processing method of this embodiment may further include the following operations: obtaining scene information of the first electronic device, the scene information including interconnection information and / or operation information of the first electronic device; and determining, based on the scene information, the target operation triggers the target interactive function of the first electronic device.

[0081] For example, scenario information can be a data set reflecting contextual features such as the current operating state of the first electronic device, the connection status of peripheral devices, and user habits. Specifically, scenario information includes at least one of interconnection information and operational information. Interconnection information can be connection status information between the first electronic device and other devices. For example, interconnection information can include a list of Bluetooth connected devices, WiFi network connection status, USB peripheral access status, Near Field Communication (NFC) pairing status, and proximity detection results of the second electronic device. For instance, when a laptop connects to both a Bluetooth headset and a Bluetooth mouse via Bluetooth, the interconnection information will record parameters such as the device identifiers, connection time, and signal strength of these two devices; when a user's mobile phone approaches the camera area of ​​the laptop, the interconnection information will include the device identifier of the mobile phone and the timestamp of the proximity event. Operational information can be data related to the application currently running on the first electronic device, system status, and hardware module operating status. For example, operational information can include the name of the application running in the foreground, the on or off status of the camera module, screen brightness level, audio output channel, system volume settings, CPU load, and network traffic. For example, when a laptop is running a video conferencing application and the camera is on, the running information will record the application's process ID and the camera's working status; when the laptop is in standby mode and the screen is off, the running information will reflect the screen's current status.

[0082] The first electronic device can acquire scene information simultaneously with or within a very short time (typically within 5 to 10 milliseconds) after detecting a target operation. Acquiring interconnection information can include reading the device connection table maintained by the operating system's Bluetooth protocol stack, querying the network status register of the WiFi driver, detecting port occupancy of the USB controller, and obtaining pairing records from the NFC chip. Acquiring runtime information can include querying the operating system's process management module to obtain the list of foreground applications, reading the status flags of the camera driver, accessing the power management module to obtain screen brightness parameters, and reading the output channel configuration of the audio driver. This acquisition of scene information typically does not require additional sensors; instead, it is achieved by reading internal state data already maintained by the operating system or hardware drivers, thus the acquisition process has minimal overhead.

[0083] After obtaining scene information, the first electronic device determines the target interactive function that the target operation should trigger based on preset scene-function mapping rules. Scene-function mapping rules can be a set of conditional judgment logic pre-configured in the first electronic device, or an intelligent recommendation model trained based on historical user data. For example, the mapping rules can be set as follows: if the interconnection information shows that multiple Bluetooth audio devices are currently connected, the target operation triggers the Bluetooth device switching function; if the operation information shows that the camera is off and the foreground application is a video conferencing application, the target operation triggers the camera quick start function; if the interconnection information shows that a second electronic device is detected approaching, the target operation triggers the cross-device file transfer function; if the operation information shows that the screen is off, the target operation triggers the screen wake-up function. This mapping rule allows the same tap or touch action to produce different responses in different scenarios, thereby achieving intelligent adaptation of one interactive operation to multiple functions.

[0084] It's worth noting that the scenario-function mapping rules can support combined judgments based on multiple scenario information. For example, when the connectivity information shows a Bluetooth headset is connected and the running information shows the foreground application is a music player, the target operation can trigger music playback or pause functions; conversely, when the connectivity information shows a Bluetooth headset is connected but the running information shows the foreground application is a browser, the target operation will trigger Bluetooth device switching or other preset functions. This combined judgment logic can be adjusted according to the user's personalized configuration to meet the usage preferences of different users.

[0085] In one example, a user is using a laptop in a work scenario. The laptop is simultaneously connected to Bluetooth headset A (for video conferencing) and Bluetooth speaker B (for music playback) via Bluetooth, with the current audio output channel set to Bluetooth headset A. The user is conducting a remote meeting with colleagues using a video conferencing application. The laptop's camera is on, the microphone is picking up the user's voice, and the screen displays the foreground interface of the conferencing application. After the meeting ends, the user wants to switch the audio output to Bluetooth speaker B to play background music for relaxation. Therefore, the user quickly taps the right palm rest area of ​​the laptop with their finger. The laptop's accelerometer detects the vibration information caused by the tap. The embedded controller (EC) determines that the vibration information meets the target conditions, activates the microphone to collect acoustic information, and after temporal alignment and feature comparison of the vibration and acoustic information, the EC confirms that the tapping action is the user's preset target operation. After confirming the target operation, EC immediately obtains the laptop's scenario information: the interconnection information shows that the current Bluetooth connected device list includes Bluetooth headset A (device address AA:BB:CC:DD:EE:01, connection duration 45 minutes, signal strength -60dBm) and Bluetooth speaker B (device address AA:BB:CC:DD:EE:02, connection duration 45 minutes, signal strength -55dBm), and the current audio output channel is Bluetooth headset A; the running information shows that the foreground application is a video conferencing application (process name "MeetingApp.exe"), the camera module is on, the system volume is 65%, and the CPU load is 35%. EC inputs the acquired scene information into a preset scene-function mapping rule engine for analysis. The mapping rule engine makes judgments based on the following logic: First, it detects multiple Bluetooth audio devices (headset A and speaker B) in the interconnection information, meeting the conditions for triggering the Bluetooth device switching function; further, combined with the running information, it finds that the foreground application is a video conferencing application and the camera is on, but since the user has just ended the meeting (inferred by detecting the change in the window focus of the conferencing application and the decrease in microphone volume), the system determines that the user's current main need is to switch audio output devices rather than camera control, thus ultimately determining the target interaction function as Bluetooth device switching. EC then executes the Bluetooth device switching process: First, it sends a command to the Bluetooth protocol stack to switch the audio output channel from Bluetooth headset A to Bluetooth speaker B; after the switch is completed, the operating system's audio routing module redirects all audio streams to Bluetooth speaker B, and the user then hears a system prompt tone through speaker B, confirming that the switch was successful.

[0086] It is understandable that by acquiring scene information of the first electronic device and dynamically determining the target interaction function triggered by the target operation based on the scene information, not only can the same physical interaction action be intelligently adapted to different functional requirements in different usage scenarios, avoiding users having to memorize multiple complex gestures or button combinations, but it can also automatically adjust the interaction response strategy based on real-time changes in interconnection information and operation information, thereby improving the convenience and intelligence level of human-computer interaction, enabling the first electronic device to more accurately understand and respond to the user's true intentions in different work or entertainment scenarios.

[0087] As described above, the processing method of this embodiment may further include the following operations: determining a target location based on first data and second data, wherein the target location represents the location information of the first operation acting on the first electronic device; and determining, based on the target location, the target operation triggers a target interactive function of the first electronic device.

[0088] For example, the target location can be the area or orientation information of the first operation on the body of the first electronic device. For instance, the target location can be different physical structural parts of the first electronic device, such as the first body (e.g., keyboard surface, palm rest area) or the second body (e.g., display screen bezel, hinge area); the target location can also be a subdivided area of ​​a specific part, such as the left bezel, right bezel, top bezel, or bottom bezel of the display screen bezel.

[0089] The first electronic device can have multiple first detection components set at different locations. For example, the first electronic device can have a first accelerometer sensor set on a first body and a second accelerometer sensor set on a second body; the first electronic device can also have multiple accelerometer sensors set at different locations on the bezel of the display screen. By analyzing the differences in the first data collected by the multiple first detection components under the first operation, the target location can be distinguished.

[0090] For example, taking a laptop computer as the first electronic device, the first body is the main body containing the keyboard, and the second body is the screen containing the display. The laptop computer has a first accelerometer sensor located on the keyboard near the palm rest, and a second accelerometer sensor located inside the bezel of the display. When a user taps the palm rest area of ​​the keyboard with their finger, the vibration intensity sensed by the first accelerometer sensor is significantly higher than that sensed by the second accelerometer sensor; conversely, when the user taps the bezel of the display, the vibration intensity sensed by the second accelerometer sensor is significantly higher than that sensed by the first accelerometer sensor. By comparing the amplitude or energy of the vibration signals collected by the first and second accelerometer sensors, it can be determined whether the first operation acts on the first body or the second body, and thus whether the target location belongs to the keyboard area or the display area.

[0091] Furthermore, the first detection component can achieve more precise position differentiation through differences in vibration information across multiple axes. For example, an accelerometer can collect acceleration data in three directions: X, Y, and Z. Vibration components in different directions exhibit different response characteristics when struck at different locations. For instance, when a user strikes the left edge of the display screen, the vibration component in the X-axis direction is relatively stronger; when the user strikes the top edge of the display screen, the vibration component in the Y-axis direction is relatively stronger. By analyzing the relative intensity or time difference of the vibration components in different axes from multiple accelerometers, it is possible to further determine whether the target location belongs to the left, right, top, or bottom edge of the display screen.

[0092] It should be noted that the target location identification in this embodiment is mainly based on the first operation acting on different structural parts or areas of the first electronic device. The distinction between the front and back sides of the same structural part (such as the screen side and the back panel side of the display screen bezel) is not specifically limited and can be adjusted according to the arrangement and sensitivity of the sensors in actual application.

[0093] The first electronic device can determine the target interactive function triggered by the target operation based on the target location and a preset mapping relationship between interactive functions. For example, the first electronic device can be pre-set to: tapping the keyboard area triggers the first interactive function, tapping the display screen area triggers the second interactive function; or, tapping the left edge of the display screen triggers the third interactive function, and tapping the top edge of the display screen triggers the fourth interactive function. The first, second, third, and fourth interactive functions can be different system functions or application operations, such as opening the camera, initiating file transfer, switching Bluetooth devices, or taking a screenshot.

[0094] In one example, a user needs to transfer files between a laptop and a mobile phone while using the laptop. The user touches the top bezel of the laptop screen with their phone. The laptop's second accelerometer detects a vibration signal as the first data, and the laptop's microphone detects an impact sound signal as the second data. Based on the fact that the vibration intensity of the second accelerometer is higher than that of the first accelerometer, and that the vibration component of the second accelerometer is relatively stronger in the Y-axis direction, the laptop determines the target location to be the top bezel area of ​​the screen. According to a preset mapping relationship, the laptop determines that the target interaction function corresponding to this target location is to initiate the file transfer process. The laptop further combines this with a tap operation detected synchronously on the mobile phone (i.e., the mobile phone's accelerometer also detects a vibration signal within a similar time), confirming that the first operation is a dual-terminal coordinated target operation, thereby triggering the file transfer function between the laptop and the mobile phone, and displaying a file selection interface for the user to select the file to be transferred.

[0095] Understandably, by setting multiple first detection components at different locations on the first electronic device and combining them with data from second detection components for comprehensive analysis, it is possible not only to accurately identify whether the first operation is the target operation, but also to determine the specific location of the target operation. This allows for the triggering of different interactive functions based on different locations, achieving a more flexible and richer human-computer interaction method and improving the user experience.

[0096] As described above, the processing method of this embodiment may further include the following operation: if the first operation is a target operation and the second electronic device is in an idle state, the third electronic device currently wirelessly connected to the first electronic device is switched to the second electronic device, and the historical connection frequency between the second electronic device and the first electronic device is greater than the historical connection frequency between the third device and the first electronic device.

[0097] For example, the second and third electronic devices can be peripheral devices capable of wireless communication with the first electronic device. For instance, the second and third electronic devices can be devices supporting Bluetooth or other wireless protocols, such as Bluetooth headsets, Bluetooth speakers, wireless keyboards, and wireless mice. The second and third electronic devices can be the same type of device, such as two different Bluetooth headsets; or they can be different types of devices, such as a Bluetooth headset and a Bluetooth speaker.

[0098] Historical connection frequency can be the number of times or the percentage of time a first electronic device establishes connections with various peripheral devices within a specific time period. For example, the first electronic device can count the number of times it has established connections with various Bluetooth devices in the past week or month; devices with more connections correspond to a higher historical connection frequency. The first electronic device can also count the cumulative connection time of each Bluetooth device; devices with longer connection times correspond to a higher historical connection frequency. Historical connection frequency can reflect a user's usage preferences or habits for different peripheral devices.

[0099] The first electronic device can exchange status information with the second electronic device via a wireless communication protocol to determine whether the second electronic device is in an idle state. For example, the first electronic device can send a status query command to the second electronic device via the Bluetooth protocol, and the second electronic device will return its current operating status information. An idle state can mean that the second electronic device has not yet established an active connection with other electronic devices, or that although a connection has been established, it is not performing primary functions such as audio playback or calls. For example, a Bluetooth headset that is paired but not playing audio can be considered in an idle state; a Bluetooth speaker that is not currently being used by other devices for audio output can be considered in an idle state. The first electronic device can also determine whether it is in an idle state by receiving status information actively broadcast by the second electronic device; for example, a Bluetooth device will periodically broadcast a connectable status indicator when idle.

[0100] For example, the first electronic device can maintain a list of paired devices, recording information such as the historical connection frequency, most recent connection time, and connection duration of each device. When the first electronic device detects a target operation, it can iterate through the list, filter out candidate devices with a historical connection frequency higher than the currently connected device, and further query the real-time status of these candidate devices to determine which devices are in an idle state as the switching target.

[0101] Furthermore, the first electronic device can determine whether to perform a device switch based on the current usage scenario. For example, if the first electronic device is currently playing music, and the second electronic device is a Bluetooth speaker that the user frequently uses in music scenarios, while the third electronic device is a Bluetooth headset that is used less frequently, then the first electronic device can switch the audio output from the third electronic device to the second electronic device after detecting the target operation; if the first electronic device is currently conducting a video conference, and the second electronic device is a Bluetooth headset that the user frequently uses in conference scenarios, then the first electronic device can switch the call audio from the third electronic device to the second electronic device after detecting the target operation.

[0102] It should be noted that the embodiments of this disclosure do not specifically limit the statistical period and calculation method for historical connection frequencies, and can be adjusted according to the usage habits analysis algorithm of the first electronic device in actual applications. The embodiments of this disclosure also do not limit the specific criteria for determining the idle state, and can be adapted according to the characteristics of different types of peripheral devices and wireless communication protocols.

[0103] In one example, a user is using a laptop for work, currently connected via Bluetooth to a regular Bluetooth headset (a third electronic device) for listening to music. The user needs to attend a video conference and wants to switch to a Bluetooth headset with better noise cancellation (a second electronic device). In the past month's usage history, the Bluetooth headset connected 30 times in video conferencing scenarios, while the regular Bluetooth headset connected only 5 times in the same scenario, indicating a significantly higher historical connection frequency for the Bluetooth headset. The user taps the palm rest area of ​​the laptop; the laptop's first accelerometer collects a vibration signal as first data, and the laptop's microphone collects an impact sound signal as second data. Based on the time and energy consistency of the first and second data, the laptop determines that the first operation is the target operation. The laptop further sends a status query command to the Bluetooth headset via Bluetooth protocol, receiving status information from the headset confirming that it is currently not actively connected to any other device and is in an idle state. According to a preset switching strategy, the laptop switches the audio output from the currently connected regular Bluetooth headset to the Bluetooth headset and displays the message "Switched to Bluetooth headset" on the screen. After the switch is complete, the audio from the video conference is played through Bluetooth headphones, eliminating the need for users to manually enter the Bluetooth settings interface to select the device, thus improving ease of operation.

[0104] Understandably, by recording and analyzing the historical connection frequency of users with different peripheral devices, and intelligently switching devices based on the real-time idle status of the devices when a target operation is detected, it can not only automatically select appropriate peripheral devices according to the user's usage habits, but also reduce the user's manual operation steps, achieving a more intelligent and personalized device management method and improving the user experience.

[0105] As described above, the processing method of this embodiment may further include the following operations: if the first operation is a target operation and / or the first operation acts on a target location of the first electronic device, the image acquisition device of the first electronic device is activated, and the interactive function of the image acquisition device is determined based on the state and / or operating information of the first electronic device.

[0106] For example, the image acquisition device can be a component in a first electronic device used to acquire image or video information. For instance, the image acquisition device can be a camera above a laptop screen, a front-facing or rear-facing camera on a tablet, or an integrated camera module in an all-in-one computer. The image acquisition device can support multiple operating modes such as still image capture, video recording, and real-time video streaming.

[0107] The status of the first electronic device can include power status, network connection status, and device interconnection status. For example, power status can include power-on, sleep, and screen-locked states; network connection status can include connected to Wi-Fi, not connected to a network, and mobile hotspot mode; device interconnection status can include establishing a wireless connection with the second electronic device, being in multi-screen collaboration mode, and being in independent working mode. The operating information of the first electronic device can include currently running applications, system resource usage, and background task execution status.

[0108] The first electronic device can assign different interactive functions to the image acquisition device based on its own state when the target operation is detected. For example, if the first electronic device is powered on and currently running a video conferencing application, the interactive function of the image acquisition device can be to activate the camera for video conferencing; if the first electronic device is in sleep mode, the interactive function of the image acquisition device can be to act as a wake-up trigger, waking up the first electronic device and putting it into working mode while activating the camera.

[0109] In one example, when the first electronic device is in sleep mode, a user may have specific usage needs when waking the device through a targeted action. After detecting the targeted action, the first electronic device can first activate an image acquisition device to perform facial recognition or environmental detection, and determine the subsequent interactive function based on the recognition results. For example, if the image acquisition device recognizes the user's facial features, the first electronic device can directly unlock and access the desktop after completing authentication; if the image acquisition device detects low ambient light, the first electronic device can automatically adjust the screen brightness after waking up. This approach extends the image acquisition device from a simple image acquisition tool to an intelligent sensing component after the device is woken up, improving the user experience when resuming from sleep mode.

[0110] Furthermore, when the first electronic device and the second electronic device are interconnected, the image acquisition device can be used as an extended camera for the second electronic device. For example, the first electronic device can be a laptop, and the second electronic device can be a smartphone. After the laptop and smartphone establish an interconnection via Wi-Fi Direct or Bluetooth, the user can perform a target operation on the laptop to activate the laptop's camera and transmit the captured image or video stream to the smartphone in real time. The smartphone can display the image captured by the laptop's camera on its screen, and the user can perform operations such as taking photos and recording videos on the smartphone. This method allows the image acquisition device of the first electronic device to be used by the second electronic device, realizing function sharing between devices.

[0111] The first electronic device can identify the current device interconnection status and the type of the second electronic device, dynamically determining the operating mode of the image acquisition device. For example, if the second electronic device is a smartphone running a camera app, the first electronic device can set the image acquisition device to remote shooting mode, directly sending the acquired image data to the smartphone's camera app for storage; if the second electronic device is a tablet computer conducting a video conference, the first electronic device can set the image acquisition device to auxiliary camera mode, providing multi-angle video feeds for the tablet computer to choose from.

[0112] It should be noted that the embodiments of this disclosure do not limit the specific interactive function types of the image acquisition device, and can be expanded according to the functions supported by the first electronic device and its collaborative capabilities with the second electronic device in actual applications. The embodiments of this disclosure also do not limit the specific protocols and connection methods for device interconnection, and can support multiple wireless communication methods such as Bluetooth, Wi-Fi Direct, and NFC.

[0113] In another example, a user is working on a document using a laptop in the office. The laptop is powered on and connected to the user's smartphone via Wi-Fi Direct. The user needs to take a picture of a paper document on their desk and send it to a colleague, but the smartphone's camera has a poor angle, while the laptop's camera is positioned better to capture the document from above. The user taps the top bezel of the laptop screen with their finger. The laptop's second accelerometer detects a vibration signal as first data, and the laptop's microphone detects an impact sound signal as second data. Based on the stronger vibration component of the second accelerometer in the Y-axis direction in the first data, the laptop determines the target location to be the top bezel area of ​​the display screen. Based on the consistency between the first and second data, the laptop determines that the first operation is the target operation. The laptop detects that it is currently connected to the smartphone and that the smartphone's camera app is running, therefore determining that the interaction function of the image acquisition device is as an extended camera for the smartphone. The laptop activates the camera and displays the message "Camera activated, providing shooting service for the phone" on the screen. The smartphone's camera app automatically switches to remote camera mode, and the screen displays the live feed captured by the laptop's camera. After adjusting the shooting angle and focus on their smartphone, users can click the shutter button. The laptop's camera then captures the image, which is directly transferred to the smartphone via Wi-Fi and saved to the photo album. Users can then send the captured images directly to colleagues from their smartphones, eliminating the need to transfer photos from the laptop to the phone first, thus simplifying cross-device collaboration.

[0114] Understandably, by dynamically determining the interactive functions of the image acquisition device based on the real-time status and operational information of the first electronic device when a target operation is detected, it not only provides intelligent wake-up and authentication capabilities when the device is in sleep mode, but also enables cross-device sharing and collaborative use of cameras in interconnected scenarios. This expands the application scenarios of the image acquisition device and enhances the convenience and flexibility of multi-device collaborative work.

[0115] As described above, in operation S220, based on the first data and the second data, it is determined whether the first operation is a target operation. In another possible implementation, the operation may further include the following operations: inputting the first data and the second data into the first model to obtain a first output result, the first output result indicating that the first operation is a target operation; or inputting the first data and the second data into the first model to obtain a second output result, the second output result indicating that the first operation is a non-target operation.

[0116] For example, the first model can be a trained machine learning model used to identify and classify different types of operations. For instance, the first model can be a neural network model, a support vector machine model, a decision tree model, or other models suitable for signal classification. The first model can be deployed on the local processor of the first electronic device or on a cloud server, with the first electronic device uploading data to the cloud for recognition and processing via a network.

[0117] The first and second output results can be the classification results of the first model after classifying the input data. For example, the first output result can be a confidence score or a classification label, indicating that the operation corresponding to the first and second input data is highly likely to be the target operation, such as a confidence score of 0.95 or a classification label of "target operation"; the second output result can indicate that the operation corresponding to the input data is not the target operation, such as a confidence score of 0.15 or a classification label of "non-target operation". The first electronic device can set a confidence threshold, such as 0.8. When the confidence score output by the first model is higher than the threshold, the first operation is determined to be the target operation; when the confidence score is lower than the threshold, the first operation is determined to be a non-target operation.

[0118] Furthermore, the first model can perform more refined identification for different operation types. For example, the first model can not only distinguish between target operations and non-target operations, but also further identify the specific type of the target operation, such as a single tap, double tap, long press, etc.

[0119] In one example, a user is using a laptop in the office. The laptop is powered on and connected to the user's smartphone via Wi-Fi Direct. The user wants to quickly transfer a file from the laptop to the smartphone. The user taps the top bezel of the laptop screen with their finger. The laptop's second accelerometer collects a vibration signal as the first data, and the laptop's microphone collects an impact sound signal as the second data. The laptop preprocesses the first data, extracting characteristic parameters such as peak acceleration, vibration duration, and vibration frequency in the X, Y, and Z axes of the vibration signal to obtain a first feature vector. The laptop preprocesses the second data, extracting characteristic parameters such as peak energy, spectral centroid, and Mel-frequency cepstral coefficients of the audio signal to obtain a second feature vector. The laptop combines the first and second feature vectors into a comprehensive feature vector, which is then input into a locally deployed first model. Based on the comprehensive feature vector, two probability values ​​are output: a target operation probability of 0.92 and a non-target operation probability of 0.08. The laptop's preset confidence threshold is 0.8. Since the target operation probability of 0.92 is higher than this threshold, the laptop determines that the first operation is the target operation and obtains the first output result. Based on the stronger vibration component detected by the second accelerometer in the Y-axis direction, the laptop determines the target location to be the upper bezel of the display screen. Combining the current connectivity with the smartphone and a pre-defined mapping, the laptop determines that the interaction triggered by this target operation will initiate a file transfer process. The laptop activates the camera and displays the message "File transfer request detected, initiating transfer service," while simultaneously displaying a file selection interface for the user to choose the file to transfer. After the user selects a file, the laptop transfers the file to the smartphone via Wi-Fi Direct.

[0120] Understandably, by inputting the first and second data into the trained first model for identification and judgment, it is possible to more accurately distinguish between target operations and non-target operations, reduce the probability of misidentification, improve the accuracy and robustness of operation detection, and provide a reliable basis for judgment for subsequent interactive function triggering.

[0121] In one possible implementation, the process of inputting the first data and the second data into the first model includes: the central processing unit (CPU) sending the first data and the second data to the embedded controller (EC); the EC forwarding the first data and the second data to the artificial intelligence chip (AICHIP); the AICHIP processing the first data and the second data based on the first model to obtain a first output result or a second output result; the AICHIP sending the first output result or the second output result to the EC; the EC triggering a system control interrupt (SCI) or a system management interrupt (SMI) based on the received first output result or the second output result; the CPU responding to the SCI or SMI interrupt reading the calculation result; when the calculation result is the first output result, the CPU executing the corresponding host service, the host service including at least one of device pairing, system unlocking, or data transmission; when the calculation result is the second output result, the CPU refusing to execute the host service or executing security protection measures.

[0122] As described above, in some embodiments, the first model can be obtained through the following operations: acquiring multiple sample data, including positive and negative samples. The positive samples are the third and fourth data corresponding to the target operation. The third data is information about the target operation acting on the first electronic device, causing the first electronic device to vibrate. The fourth data is acoustic information generated by the target operation acting on the first electronic device. The negative samples are the fifth and sixth data corresponding to non-target operations. The fifth data is information about the non-target operation acting on the first electronic device, causing the first electronic device to vibrate. The sixth data is acoustic information generated by the non-target operation acting on the first electronic device. The target operation and non-target operation are the true labels of the sample data. The sample data is input into the first model to obtain prediction results. The prediction results are operation labels corresponding to the sample data. The operation labels include the target operation and non-target operation. The parameters of the first model are updated according to the difference between the prediction results and the true labels until the first model converges, thus obtaining the trained first model.

[0123] For example, the sample data can be sensor data and acoustic data corresponding to various operations collected during the actual use of the first electronic device. For instance, a dedicated sample acquisition program can be used to have multiple users execute a preset sequence of operations on multiple first electronic devices, recording the accelerometer data and microphone audio data corresponding to each operation, while also recording the actual type of each operation as a label. Sample data can also be obtained through simulation experiments, such as using a robotic arm or other automated equipment to perform standardized tapping, touching, or other operations on the first electronic device, while simultaneously recording sensor response data.

[0124] Positive samples can include data corresponding to various target operation types. For example, positive samples can include third and fourth data generated when a user taps the top bezel of the display screen with their finger, data generated when a user taps the side of the device, and data from various predefined target operations such as double-tapping the touchpad or long-pressing a specific button. The third data can include information such as the acceleration time-series curves, peak values, frequency, and duration of the vibration generated by the tapping in three axes. The fourth data can include information such as the time-domain waveform, spectral distribution, audio energy, and duration of the impact sound generated by the tapping.

[0125] Negative samples can include data corresponding to various non-target operations. For example, negative samples can include fifth and sixth data generated by the user moving the first electronic device, data generated by the first electronic device being placed on a vibrating table, and data generated by interference scenarios such as the user typing on a keyboard or other objects colliding in the environment. The fifth data can include vibration information generated by non-target operations such as device movement and table vibration, which differ from the third data generated by the target operation in terms of vibration mode, frequency characteristics, and duration. The sixth data can include acoustic information that differs from the sound characteristics of the target operation, such as ambient noise and keyboard clicks.

[0126] The prediction result can be the classification result output by the first model after processing the input sample data. For example, the first model can output a probability distribution for each sample data, representing the probability that the sample belongs to the target operation and not the target operation, and then determine the operation label based on the category with the higher probability. If the first model outputs the prediction label "target operation" for a positive sample, the prediction is correct; if the output prediction label is "non-target operation", the prediction is incorrect.

[0127] The parameters of the first model are updated based on the difference between the predicted results and the true labels. Backpropagation and gradient descent can be used. For example, a loss function can be defined to quantify the difference between the predicted results and the true labels, such as the cross-entropy loss function. The loss function is larger when the predicted results are inconsistent with the true labels and smaller when the predicted results are consistent with the true labels. By calculating the gradient of the loss function with respect to the parameters of each layer of the first model, the model parameters are adjusted along the direction of gradient descent, gradually reducing the value of the loss function. After multiple rounds of iterative training, the prediction accuracy of the first model gradually improves.

[0128] The convergence of the first model indicates that the model training has reached a stable state, and further training will have limited effect on improving model performance. For example, the convergence condition can be set as follows: the value of the loss function changes by less than a preset threshold over several consecutive training rounds, or the recognition accuracy on the validation set no longer improves significantly. When the convergence condition is met, the training process is terminated, and the first trained model is obtained.

[0129] As described above, in operation S220: determining whether the first operation is a target operation based on the first data and the second data, in one possible implementation, this operation may further include: acquiring third data from the second electronic device, the third data being vibration information generated by the first operation acting on the second electronic device; and determining whether the first operation is a target operation based on the first data, the second data, and the third data.

[0130] For example, the second electronic device can be another electronic device that is interconnected or physically close to the first electronic device. For instance, the second electronic device can be a smartphone, tablet, smartwatch, or other device that establishes a communication connection with the laptop via Bluetooth or Wi-Fi, or it can be another electronic device placed on the same table as the laptop, although not establishing a communication connection, but spatially close. The second electronic device can be equipped with motion sensors such as accelerometers and gyroscopes, capable of detecting physical changes such as vibration and movement experienced by the device.

[0131] The third data may include acceleration time-series data, vibration peak value, vibration frequency, vibration duration, and other information collected by the accelerometer of the second electronic device in three axes.

[0132] Determining whether a first operation is the target operation based on the first, second, and third data can employ a multi-dimensional signal correlation analysis method. For example, the first electronic device can analyze the temporal synchronicity and signal characteristic consistency of the first data (vibration information of the first electronic device itself), the second data (acoustic information), and the third data (vibration information of the second electronic device) to comprehensively judge the authenticity and validity of the first operation. If the three sets of data highly match in timestamps, and both vibration and acoustic characteristics conform to the typical pattern of the target operation, then the first operation is determined to be the target operation; if there are significant time differences or feature mismatches among the three sets of data, then the first operation is determined to be a non-target operation.

[0133] In one example, the first electronic device can extract the temporal characteristics of the vibration signal from the first data, such as the vibration start time T1 and the vibration peak occurrence time T2. Simultaneously, the first electronic device can extract the temporal characteristics of the acoustic signal from the second data, such as the sound start time T3 and the sound peak occurrence time T4. The first electronic device can also extract the vibration temporal characteristics detected by the second electronic device from the third data, such as the vibration start time T5 and the vibration peak occurrence time T6. If the time differences between times T1, T3, and T5 are all within a preset time window (e.g., 50 milliseconds), and the time differences between times T2, T4, and T6 are also within the preset time window, it indicates that the three sets of data are highly synchronized in time, and the first operation is likely the actual target operation. If time T5 differs significantly from times T1 and T3 (e.g., more than 500 milliseconds), it indicates that the vibration detected by the second electronic device and the vibration and sound detected by the first electronic device are not caused by the same operation, and the first operation may be an interference signal or a non-target operation.

[0134] Understandably, by acquiring third data from the second electronic device as an auxiliary basis for judgment, and conducting multi-dimensional correlation analysis between the vibration and acoustic information of the first electronic device and the vibration information of the second electronic device, it is possible to more accurately identify the user's true operating intention, effectively eliminate interference factors such as environmental vibration and device movement, and improve the accuracy and reliability of target operation judgment. This is particularly suitable for operation recognition in mobile scenarios and complex environments, further enhancing the intelligence level and user experience of electronic device interaction systems.

[0135] A comparison of the processing method of this disclosure embodiment with related technologies yields the following results:

[0136]

[0137] Based on the above processing method, this disclosure also provides a first electronic device. The following will be combined with... Figure 3 The device is described in detail.

[0138] Figure 3 A schematic diagram of a first electronic device according to an embodiment of the present disclosure is shown.

[0139] like Figure 3 As shown, the first electronic device 100 of this embodiment includes at least one first detection component 110, at least one second detection component 120, and a processor 130.

[0140] At least one first detection component 110 is used to acquire first data from the first electronic device 100 in response to a first operation.

[0141] At least one second detection component 120 is used to acquire second data from the first electronic device 100 in response to the first operation.

[0142] The processor 130 is configured to respond to a first operation by obtaining first data through a first detection component 110 of the first electronic device 100 and second data through a second detection component 120 of the first electronic device 100. The first detection component 110 and the second detection component 120 are located at different positions in the first electronic device 100 and are detection components with different functions. Based on the first data and the second data, the processor 130 determines whether the first operation is a target operation. The target operation is used to control the first electronic device 100 to perform a function corresponding to the target operation.

[0143] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a processing method according to an embodiment of the present disclosure.

[0144] like Figure 4 As shown, an electronic device 400 according to an embodiment of the present disclosure includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage portion 408 into a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0145] RAM 403 stores various programs and data required for the operation of electronic device 400. Processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Processor 401 executes various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 402 and / or RAM 403. It should be noted that programs may also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 may also execute various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0146] According to embodiments of this disclosure, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to a bus 404. The electronic device 400 may also include one or more of the following components connected to the I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0147] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0150] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A processing method applied to a first electronic device, the method comprising: In response to the first operation, first data is obtained through the first detection component of the first electronic device, and second data is obtained through the second detection component of the first electronic device. The first detection component and the second detection component are located at different positions in the first electronic device and are detection components with different functions. Based on the first data and the second data, it is determined whether the first operation is a target operation, wherein the target operation is used to control the first electronic device to perform a function corresponding to the target operation.

2. The method according to claim 1, wherein the first detection component is used to collect vibration information generated by the first electronic device due to the first operation, and the second detection component is used to collect acoustic information generated by the first operation.

3. The method according to claim 1 or 2, in response to the first operation, obtaining first data through a first detection component of the first electronic device, and obtaining second data through a second detection component of the first electronic device, includes: In response to the first operation, first data is obtained through the first detection component of the first electronic device; If the first data meets the target condition, the second data is obtained through the second detection component of the first electronic device, wherein the target condition characterizes the vibration information of the first electronic device in a vibration mode.

4. The method according to claim 1, further comprising: Obtain scene information of the first electronic device, the scene information including interconnection information and / or operation information of the first electronic device; Based on the scenario information, it is determined that the target operation triggers the target interaction function of the first electronic device.

5. The method according to claim 1 or 4, further comprising: Based on the first data and the second data, a target location is determined, wherein the target location represents the location information of the first operation acting on the first electronic device; Based on the target location, the target operation is determined to trigger the target interactive function of the first electronic device.

6. The method according to claim 1, further comprising: If the first operation is a target operation and the second electronic device is in an idle state, the third electronic device currently wirelessly connected to the first electronic device is switched to the second electronic device, and the historical connection frequency between the second electronic device and the first electronic device is greater than the historical connection frequency between the third device and the first electronic device.

7. The method according to claim 1, further comprising: If the first operation is a target operation and / or the first operation is applied to a target location of the first electronic device, the image acquisition device of the first electronic device is activated, and the interactive function of the image acquisition device is determined based on the status and / or operating information of the first electronic device.

8. The method according to claim 1, wherein determining whether the first operation is a target operation based on the first data and the second data includes: The first data and the second data are input into the first model to obtain the first output result. The first output result indicates that the first operation is the target operation. or The first data and the second data are input into the first model to obtain the second output result. The second output result indicates that the first operation is a non-target operation.

9. The method according to claim 1, wherein determining whether the first operation is a target operation based on the first data and the second data includes: Acquire third data from the second electronic device, wherein the third data is vibration information generated by the second electronic device due to the first operation acting on it; Based on the first data, the second data, and the third data, determine whether the first operation is the target operation.

10. A first electronic device, comprising: At least one first detection component is configured to acquire first data from the first electronic device in response to a first operation; At least one second detection component is used to acquire second data from the first electronic device in response to the first operation; A processor is configured to, in response to a first operation, obtain first data through a first detection component of the first electronic device and obtain second data through a second detection component of the first electronic device, wherein the first detection component and the second detection component are located at different locations in the first electronic device and are detection components with different functions; and, based on the first data and the second data, determine whether the first operation is a target operation, wherein the target operation is used to control the first electronic device to perform a function corresponding to the target operation.

Images