Interaction method and electronic equipment

Abstract

The invention provides an interaction method and electronic equipment, and relates to the technical field of electronic equipment. The interaction method comprises the following steps: when an interaction control mode is a first interaction control mode, acquiring first control environment information corresponding to the first interaction control mode in a target control environment; if the first control environment information does not meet the first control condition, switching the interactive control mode to a second interactive control mode; wherein the first interaction control mode is different from the second interaction control mode, and in the target control environment, the control success rate based on the first interaction control mode is smaller than the control success rate based on the second interaction control mode.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic devices, and in particular, to an interaction method and an electronic device. Background Art

[0002] With the wide application of interactive systems, the interaction control methods between users and electronic devices have become increasingly diversified. However, in actual applications, the interaction control methods often cannot meet the requirements of complex control environments, resulting in a low success rate of user control of electronic devices in complex control environments. Summary of the Invention

[0003] In view of this, the present disclosure provides an interaction method and an electronic device.

[0004] According to a first aspect of the present disclosure, there is provided an interaction method, including:

[0005] When the interaction control method is a first interaction control method, obtaining first control environment information corresponding to the first interaction control method in a target control environment;

[0006] If the first control environment information does not meet a first control condition, switching the interaction control method to a second interaction control method;

[0007] Wherein, the first interaction control method is different from the second interaction control method, and in the target control environment, the control success rate based on the first interaction control method is less than the control success rate based on the second interaction control method.

[0008] According to an embodiment of the present disclosure, a first control information acquisition sensor for collecting control information in the first interaction control method is different from a second control information acquisition sensor for collecting control information in the second interaction control method.

[0009] According to an embodiment of the present disclosure, the switching the interaction control method to the second interaction control method includes:

[0010] Starting the second control information acquisition sensor, and / or, closing the first control information acquisition sensor.

[0011] According to an embodiment of the present disclosure, a first control information input method of a user in the first interaction control method is the same as or different from a second control information input method of the user in the second interaction control method.

[0012] According to an embodiment of the present disclosure, the first control environment information includes environmental audio information of the target control environment, and the first control information input method is information input by voice.

[0013] The fact that the first control environment information does not meet the first control condition indicates that the audio intensity of the environmental audio in the target control environment is greater than the first audio intensity threshold, and / or the audio intensity of the audio emitted by the target object in the target control environment is less than the second sound intensity threshold.

[0014] According to an embodiment of the present disclosure, the method further includes:

[0015] Perform frequency division on the audio information collected by the first control acquisition sensor to obtain the environmental audio of the target control environment and the audio emitted by the target object.

[0016] According to an embodiment of the present disclosure, the input method of the first control information of the user in the first interactive control mode is different from the input method of the second control information of the user in the second interactive control mode;

[0017] The method further includes:

[0018] Output a prompt message indicating a switch to the second interactive control mode.

[0019] According to an embodiment of the present disclosure, both the first interactive control mode and the second interactive control mode are non-contact interactive control modes.

[0020] According to an embodiment of the present disclosure, the first control environment information is the information collected by the first control information acquisition sensor;

[0021] The method further includes:

[0022] Obtain the target data collected by the first control information acquisition sensor;

[0023] If the first control environment information included in the target data does not meet the first control condition, switch the interactive control mode to the second interactive control mode;

[0024] If the first control environment information included in the target data meets the first control condition, based on the control information included in the target data, perform a control operation corresponding to the control information.

[0025] A second aspect of the present disclosure provides an electronic device, including:

[0026] A first control information acquisition sensor, configured to collect control information in the first interactive control mode;

[0027] A second control information acquisition sensor, configured to collect control information in the second interactive control mode;

[0028] One or more processors, configured to execute:

[0029] When the interaction control mode is the first interaction control mode, obtain the first control environment information corresponding to the first interaction control mode in the target control environment; if the first control environment information does not meet the first control condition, switch the interaction control mode to the second interaction control mode; wherein, the first interaction control mode is different from the second interaction control mode, and in the target control environment, the control success rate based on the first interaction control mode is less than the control success rate based on the second interaction control mode.

[0030] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, the above and other objects, features and advantages of the present disclosure will become clearer. In the drawings:

[0032] Figure 1 Schematically shows a flowchart of an interaction method provided by an embodiment of the present disclosure;

[0033] Figure 2 Schematically shows a scenario diagram of switching of an interaction control mode provided by an embodiment of the present disclosure;

[0034] Figure 3 Schematically shows a scenario diagram of sensor switching provided by an embodiment of the present disclosure;

[0035] Figure 4 Schematically shows a block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the purpose of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0037] The terms used herein are merely for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0038] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0039] In cases where expressions similar to "at least one of A, B, and C" are used, generally, it should be interpreted according to the meaning that those of ordinary skill in the art usually understand such expressions (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0040] In the embodiments of the present disclosure, in aspects such as the collection, update, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the involved data (for example, including but not limited to user personal information), they all comply with the provisions of relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and to safeguard the security of user personal information, network security, and national security.

[0041] The embodiments of the present disclosure provide an interaction method and an electronic device. Among them, the interaction method includes: when the interaction control method is the first interaction control method, obtaining the first control environment information corresponding to the first interaction control method in the target control environment; if the first control environment information does not meet the first control condition, switching the interaction control method to the second interaction control method; where the first interaction control method is different from the second interaction control method, and in the target control environment, the control success rate based on the first interaction control method is less than the control success rate based on the second interaction control method.

[0042] By adopting the embodiments of the present disclosure, the first control environment information corresponding to the first interaction control method in the target control environment is obtained, and conditional judgment is made based on this information, realizing the perception of the environment by the interaction system. When the first control environment information does not meet the first control condition, it is determined that the current first interaction control method is not suitable for continued use, and at this time, it is automatically switched to the second interaction control method with a higher control success rate in the target control environment. The above interaction method based on the control environment can ensure that the electronic device always adopts the most suitable control method for the current environment, thereby improving the control success rate of the user for the electronic device in a complex environment.

[0043] The electronic device in the embodiments of the present disclosure refers to a device capable of realizing human-computer interaction. Such electronic devices include mobile terminals such as smart phones, tablet computers, laptop computers, smart watches, smart earphones, smart glasses, etc., as well as human-computer interaction devices such as augmented reality (AR) / virtual reality (VR) devices, wearable computing devices, smart speakers, etc. In addition, intelligent terminals supporting various interaction methods such as in-vehicle intelligent systems, Internet of Things (IoT) terminals, health monitoring devices, and smart home controllers, as well as computing devices such as servers, personal computers (PCs), and industrial control devices, also belong to the category of electronic devices described in the embodiments of the present disclosure. At the same time, wearable electronic devices equipped with various sensors, intelligent control systems supporting environmental perception, and other devices are also within the application scope of the embodiments of the present disclosure.

[0044] It should be noted that the embodiments of the present disclosure do not limit the specific types of electronic devices. The above examples are only illustrative descriptions, and the technical solutions of the present disclosure can be applied to any electronic device with various interactive control methods and capable of sensing the control environment. The present disclosure is particularly applicable to various electronic devices that need to maintain a high control success rate in special control environments such as noisy environments, complex light conditions, and changes in the user's activity state.

[0045] The following Figures 1 to 3 describes the interaction method of the embodiments of the present disclosure in detail.

[0046] Figure 1 Schematically shows a flowchart of an interaction method provided by the embodiments of the present disclosure.

[0047] Operation S101, when the interactive control method is the first interactive control method, obtain the first control environment information corresponding to the first interactive control method in the target control environment;

[0048] Operation S102, if the first control environment information does not meet the first control condition, switch the interactive control method to the second interactive control method; wherein, the first interactive control method is different from the second interactive control method, and in the target control environment, the control success rate based on the first interactive control method is less than the control success rate based on the second interactive control method.

[0049] In operation S101, the interactive control method refers to the specific way of information exchange and operation control between the user and the electronic device. In the embodiments of the present disclosure, it can be understood as the entire process and mechanism in which the user sends instructions to the electronic device through a specific input method, and the electronic device receives, analyzes, and executes the instructions. Different interactive control methods are applicable to different control environments and have their own advantages and limitations.

[0050] Exemplarily, the interactive control methods include but are not limited to: various forms such as voice control, touch operation, gesture recognition, eye movement tracking, head movement recognition, and button input.

[0051] Furthermore, the first interactive control method refers to the interactive control method currently applied by the electronic device. The determination of the first interactive control method can be achieved through various means: firstly, it can be determined by the default interactive settings of the electronic device. For example, smart headphones default to voice control, and smart glasses default to eye movement tracking; secondly, the user can explicitly select it through the interface, shortcut keys, or specific commands provided by the device; thirdly, the system can make an intelligent recommendation based on historical usage data and automatically select the interactive method that the user most frequently uses or has the highest control success rate in a similar environment.

[0052] On this basis, the target control environment refers to the specific environment in which the user is currently interacting with the electronic device. In the embodiments of the present disclosure, it can be understood as the comprehensive environment that affects the physical conditions, external factors, and user status of the user's control of the electronic device. The target control environment is used to evaluate the applicability of the current interactive control method and determine whether it is necessary to switch to a more suitable method. When the target control environment changes, it may lead to a decrease in the control success rate of the original interactive control method, thereby triggering the adaptive adjustment of the interactive control method.

[0053] Correspondingly, the first control environment information refers to the environmental parameters and condition data related to the first interactive control method. In the embodiments of the present disclosure, it can be understood as the quantitative representation of the key environmental factors affecting the effectiveness of the first interactive control method and their current states. The first control environment information is used to evaluate the control success rate and applicability of the first interactive control method in the current target control environment.

[0054] According to different first interactive control methods, the electronic device needs to collect the first control environment information that matches them. The electronic device collects these environmental data through various sensors to accurately evaluate the current applicability of the first interactive control method.

[0055] Exemplarily, the voice control method mainly focuses on environmental noise and human voice interference; touch operation focuses on the user's hand state and screen conditions; gesture recognition focuses on light brightness and field of view conditions; eye movement tracking needs to monitor light conditions and the user's head stability.

[0056] Specifically, the electronic device first confirms the currently used first interaction control method, and then determines the first control environment information to be collected based on the first interaction control method. Subsequently, the sensor combination corresponding to the first control environment information is activated to collect the first control environment information in a targeted manner.

[0057] Figure 2 Schematically shows a scenario diagram of switching of an interaction control method provided by an embodiment of the present disclosure.

[0058] In a practical application scenario, as Figure 2 shown, the user wears the electronic watch 201 and uses the touch operation 202 as the first interaction control method for daily interaction. When the user is about to swim, the electronic watch 201 starts to execute operation S101. The system first confirms that the currently used interaction control method is the touch operation 202. Subsequently, based on the characteristics of the touch operation 202, the system determines that the key environmental parameters to be monitored include the humidity on the surface of the touch screen, the activity state of the user's hand, and the changes in environmental temperature and humidity, etc. Accordingly, the system activates a combination of a humidity sensor, an acceleration sensor, a gyroscope, and a temperature sensor on the device.

[0059] When the user enters the swimming pool area, the humidity sensor of the device starts to record that the humidity value on the surface of the touch screen rapidly rises from the normal 10% to 85%, indicating that the screen surface is largely covered by moisture; at the same time, the acceleration sensor and the gyroscope detect that the user's arm starts to perform regular swimming strokes, presenting a typical freestyle swimming mode; the temperature sensor also detects that the environmental temperature changes from 24°C in the office to 28°C in the pool area, and the relative humidity rises from 45% to 90%. The system analyzes these raw data through the signal processing algorithm built into the device to identify the current first control environment information. In the current swimming environment, the touch screen is largely covered by moisture, and the user's hand is in a high-frequency periodic motion. The first control environment information will affect the control success rate of the touch operation 202.

[0060] On this basis, the first control environment information provides a basis for the system to subsequently judge the applicability of the touch operation 202 in the target control environment, enabling the electronic watch 201 to actively evaluate the limitations of the first interaction control method before the user attempts to control the electronic watch 201 through the touch operation 202.

[0061] In operation S102, the system first needs to determine whether the first control environment information meets the first control condition, so as to provide a basis for subsequent interaction mode switching. Among them, the first control condition refers to the environmental parameter standard for evaluating the applicability of the first interactive control mode. In the embodiments of the present disclosure, it can be understood as a preset set of thresholds or judgment criteria for determining whether the current target control environment is suitable for continuing to use the first interactive control mode. The first control condition usually consists of environmental parameter thresholds closely related to the first interactive control mode.

[0062] Exemplarily, when the first interactive control mode is voice control, the first control condition may include: the ambient noise level does not exceed 65 dB, the user voice signal strength is not lower than 45 dB, the ambient echo coefficient is lower than 0.3, and there is no obvious interfering sound source around, etc. When the first control environment information exceeds the first control condition, the control success rate of voice control will decrease significantly, and the system needs to consider switching to an interactive mode that is more adapted to the target control environment.

[0063] It should be noted that the first control condition can be a statically preset fixed standard or a dynamically adjusted adaptive standard. The static standard is applicable to most standard usage scenarios, while the dynamic standard will be adaptively adjusted according to user habits, personal preferences, and specific application scenario requirements to provide a more personalized interaction experience. For example, the system may dynamically increase the ambient noise threshold for voice control for specific users with better-than-average voice recognition capabilities.

[0064] When the system determines that the first control environment information does not meet the first control condition, it is necessary to switch the interactive control mode to the second interactive control mode. Among them, the second interactive control mode refers to the alternative interactive mode switched by the system. In the embodiments of the present disclosure, it can be understood as an interactive control mode with a higher control success rate selected by the system based on the characteristics of the current target control environment, which is used to ensure that the user can still interact with the electronic device efficiently when the target control environment changes.

[0065] Similarly, the determination of the second interactive control mode is the optimal choice after the system evaluates the applicability of multiple available interactive modes in the current target control environment, rather than being randomly determined. It comprehensively considers multi-dimensional factors such as the target control environment information, historical interaction data, user preference settings, and the current state of the device to ensure that the selected mode is most suitable for the current control environment.

[0066] In different target control environments, the second interactive control method may be manifested as different specific interactive forms. Exemplarily, when the first interactive control method is voice control and the target control environment is a high-noise environment, the second interactive control method may be eye tracking; when the first interactive control method is touch operation and the target control environment is a state where the hands are busy, the second interactive control method may be head gesture control; when the first interactive control method is gesture recognition and the target control environment is a low-light environment, the second interactive control method may be voice control.

[0067] For a specific type of electronic device, the system will preset multiple candidate second interactive control methods according to its hardware capabilities and application scenarios. For example, smart glasses can preset multiple interactive methods such as eye tracking, head gesture, voice control, and touch operation; smart watches can preset interactive methods such as touch operation, gesture recognition, voice control, and wrist movement; in-vehicle systems can preset interactive methods such as voice control, steering wheel buttons, touch screens, and head gestures. The system will select one of these preset candidate methods that is most suitable for the current target control environment as the second interactive control method.

[0068] It should be noted that the second interactive control method in the embodiments of the present disclosure is not static, but can be dynamically adjusted according to the continuous change of the target control environment. When the target control environment changes again, the current second interactive control method will become the new first interactive control method, and the system will re-evaluate its control success rate and switch to a new second interactive control method when necessary, forming a dynamic interactive system that continuously adapts to the target control environment.

[0069] Furthermore, in order to ensure that the switched second interactive control method is indeed better than the first interactive control method, the embodiments of the present disclosure provide multiple methods for comparing the control success rates of different interactive control methods:

[0070] In a feasible implementation manner, the system can analyze the success rates of different interactive control methods through historical data. The system continuously records the operation records of users using various interactive methods in different environments in the background, including information such as operation instructions, environmental parameters, operation results, and response times. When it detects that the current environment is similar to the historical environment, the system can directly query and predict the control success rates of each interactive method. For example, when it detects that the environmental noise reaches 80 dB, the system queries the historical data and may find that in this environment, the success rate of voice control is 45%, while the success rate of eye movement control is 85%.

[0071] In another feasible implementation, the system can compare the success rates of different control methods through real-time interaction quality assessment. The electronic device monitors the key performance indicators of the current first interaction control method in real time (such as command recognition rate, response time, etc.), and at the same time predicts the performance indicators of other available interaction methods in the current environment. Based on the above performance indicators, the system can calculate the expected control success rate of each interaction method. For example, in a strong light environment, the system may detect that the success rate of eye tracking drops to 40%, while the predicted success rate of voice control is 85%.

[0072] In yet another feasible implementation, the system can compare the success rates of different control methods through an intelligent model or algorithm model related to environmental factors. The electronic device pre-stores the sensitivity models of various interaction control methods to different environmental factors, and this sensitivity model quantifies the degree of influence of environmental parameter changes on the success rate of each interaction method. When the target control environment parameters are obtained, the system calculates the expected control success rate of each interaction method in the current environment through the sensitivity model. For example, the model may show that when the hand jitter increases by 30%, the success rate of touch operation drops to 55%, while the success rate of voice control remains at 94%.

[0073] Through the above methods for comparing the control success rates of different interaction control methods, the system can ensure that when the first control environment information does not meet the first control condition, the switched-to second interaction control method has a higher control success rate, thereby improving the user's device control experience in complex environments.

[0074] For the application scenario exemplified by operation S101, the system executes the following processing flow in operation S102:

[0075] First, after the electronic watch 201 system obtains the first control environment information, it immediately compares it with the preset first control condition. For the touch operation 202, the first control conditions preset by the system include: multiple standards such as the humidity on the surface of the touch screen not exceeding 30% and the amplitude of the user's hand jitter being less than 2 mm / s. After system analysis, it is obtained that in the current environment, the humidity on the surface of the touch screen is as high as 85%, far exceeding the threshold standard of 30%; the user's arm is making regular swimming motions, and the amplitude of hand jitter reaches 8 mm / s, also exceeding the preset threshold.

[0076] Based on the above analysis, the system determines that the first control environment information clearly does not meet the first control condition, triggering the switching process of the interactive control method. The electronic watch 201 first evaluates the expected control success rate of each candidate interactive method in the current swimming environment. The system can analyze by querying historical data that in a similar high-humidity and high-frequency hand movement environment, the average control success rate of the touch operation 202 is only 15%, while the voice control has a success rate of about 30% due to being easily interfered by water sounds and breathing, and the wrist movement recognition 203, which is optimized specifically for the sports scenario, has an expected control success rate as high as 85% in the swimming environment.

[0077] As Figure 2 shown, the system immediately switches the interactive control method from the touch operation 202 (the first interactive control method) to the wrist movement recognition 203 (the second interactive control method). A prompt of "Switched to wrist movement control mode" is briefly displayed on the screen of the electronic watch 201, and the user is notified of the change in the interactive method through a slight vibration feedback. The system simultaneously activates the algorithm module dedicated to recognizing specific wrist movements in the swimming environment and starts monitoring the user's specific wrist torsion, stay, and flipping movements.

[0078] In the wrist movement recognition 203 interactive method, the user can control the watch through preset specific wrist movements without touching the screen. For example, the user can view the current swimming distance by quickly flipping the wrist upwards twice in a row; start the heart rate monitoring by keeping the wrist horizontal and rotating it counterclockwise by 90 degrees. The specific wrist movements are designed to be natural and feasible during swimming and not easily confused with normal swimming movements, improving the success rate of controlling the electronic watch 201 in water.

[0079] When the user finishes swimming, leaves the pool area, and dries the watch and hands, the system can execute operation S101 again and detects that the environmental conditions have returned to a state suitable for the touch operation 202: the humidity on the surface of the touch screen has dropped to 15%, the hand movement has returned to normal, and the contact stability between the device and the skin has risen to 90%. At this time, the system determines that the environmental conditions have re-met the first control condition of the touch operation 202, so it switches the wrist movement recognition 203 (as the first interactive control method) back to the touch operation 202 (the new second interactive control method), and at the same time displays a prompt of "Restored touch control mode", completing the adaptive adjustment of the interactive control method.

[0080] By adopting the embodiments of the present disclosure, the first control environment information corresponding to the first interactive control method in the target control environment is obtained, and conditional judgment is performed based on this information, realizing the perception of the environment by the interactive system. When the first control environment information does not meet the first control condition, it is determined that the current first interactive control method is not suitable for continued use, and at this time, it is automatically switched to the second interactive control method with a higher control success rate in the target control environment. The above interactive method based on the control environment can ensure that the electronic device always adopts the most suitable control method for the current environment, thereby improving the control success rate of the user for the electronic device in a complex environment.

[0081] During the interaction between the user and the electronic device, when the user is in a special environment or a specific situation, the traditional contact-based interaction method may face many limitations, resulting in a decrease in control efficiency and user experience. For example, in a medical surgical environment, doctors need to keep their hands sterile and cannot directly touch the electronic device to view or input data.

[0082] Although the related technologies have provided a variety of contactless interaction methods, a single contactless interaction method is usually highly sensitive to specific environmental factors and is easily affected by environmental changes and fails. For example, gesture recognition is difficult to accurately capture the user's intention in an environment with insufficient light or limited space.

[0083] To solve the above technical problems, on the basis of the above embodiments, as an optional embodiment, the first interactive control method and the second interactive control method provided in the embodiments of the present disclosure can both be contactless interactive control methods.

[0084] Among them, the contactless interactive control method refers to a control method in which the user can complete the interactive operation without directly contacting the physical interface of the device, including but not limited to voice control, eye movement tracking, head gesture control, gesture recognition, and wrist movement recognition 203, etc.

[0085] Specifically, the system will first select one of the various available contactless interaction methods as the first interactive control method according to the user's current target control environment.

[0086] Taking a medical operating room as an example, doctors usually have a relatively clear voice environment at the beginning of the operation, and voice control can provide convenient interaction while the doctor's hands are focused on the operation. Therefore, the system sets voice control as the first interactive control method, and the surgeon controls the display device in the operating room to view the patient's imaging data or vital sign data through oral instructions.

[0087] When the system performs operation S101, the system continuously monitors and collects the first control environment information of voice control in the current operating room environment, including environmental noise levels (such as the running sound of surgical instruments, the alarm sound of monitoring equipment), voice recognition quality parameters (such as voice clarity, instruction integrity), and environmental conversation frequency (such as the professional communication frequency between medical staff), etc.

[0088] When the operation enters a critical stage and the target control environment changes significantly, such as when multiple medical staff are talking simultaneously or multiple medical devices are emitting prompt sounds at the same time, the environmental noise level may exceed the preset first control condition. At this time, the system determines that the first control environment information does not meet the first control condition and triggers the execution of operation S102.

[0089] To determine the optimal second interaction control method, the system evaluates the control success rates of all available contactless interaction methods in the current environment. For example, in a noisy operating room environment, the expected success rate of voice control has dropped to 40%, while for eye tracking, since it is not affected by environmental noise and doctors' line of sight often focuses on the display device, its expected success rate remains above 90%; gesture recognition is limited by the fact that doctors' hands need to perform surgical operations, and the expected success rate is only 30%. Based on the above analysis, the system determines eye tracking as the second interaction control method with the highest control success rate.

[0090] The system then executes the interaction method switching operation, switching the interaction control method from voice control to eye tracking. To ensure the smoothness of the switching process, the system will display a short visual prompt at the edge of the display screen to inform the doctor that the current mode has been switched to eye tracking mode. The eye tracking system starts to work, capturing the doctor's eye movements through the built-in infrared camera and converting them into control commands.

[0091] By adopting the embodiments of the present disclosure, both the first interaction control method and the second interaction control method are set as contactless interaction control methods. In a special environment where users cannot use traditional contact-based interactions, at least one interaction method suitable for the current environment can still be found to effectively interact with the device. Thus, a high control success rate can be maintained in various complex environments.

[0092] In related electronic device interaction systems, different types of interaction control methods rely on specific types of sensors for information collection, and different sensors have different sensitivities to environmental conditions. Electronic devices are usually equipped with multiple sensors to support different interaction methods, but lack an effective sensor switching mechanism.

[0093] To solve the above technical problems, based on the above embodiments, as an alternative embodiment, the embodiments of the present disclosure provide a first manipulation information acquisition sensor for acquiring manipulation information in a first interaction manipulation mode, and a second manipulation information acquisition sensor for acquiring manipulation information in a second interaction manipulation mode, wherein the types of the first manipulation information acquisition sensor and the second manipulation information acquisition sensor are different.

[0094] In a feasible implementation, the first manipulation information acquisition sensor and the second manipulation information acquisition sensor can be disposed in the same electronic device. For example, a smart glasses may be built-in with a variety of sensors such as a microphone array, an eye tracking camera, a touchpad, an accelerometer, and a gyroscope. When the environmental conditions change, the system can switch between the existing sensors inside the device without relying on external devices.

[0095] In another feasible implementation, the first manipulation information acquisition sensor can be disposed in a first electronic device, and the second manipulation information acquisition sensor can be disposed in a second electronic device communicatively connected to the first electronic device. The above distributed sensor design is applicable to the Internet of Things environment and multi-device collaboration scenarios. For example, the smart watch worn by a user may mainly rely on the built-in touch screen sensor for interaction, but when the user enters a specific environment, the system may switch to the microphone array on the speaker near the user or the camera on the TV to acquire manipulation information.

[0096] In yet another feasible implementation, considering that a single sensor often fails to provide sufficiently accurate information acquisition in a complex environment, the first manipulation information acquisition sensor can be a single sensor, and the second manipulation information acquisition sensor can be a combination of multiple sensors working together. Multi-sensor fusion can improve the overall perception accuracy through complementary advantages. For example, under normal lighting conditions, the system may only use an ordinary camera to recognize user gestures; but when the light becomes dim, the system may switch to a combination of a depth camera and an infrared camera.

[0097] It should be noted that the above embodiments are not mutually exclusive, but can be flexibly combined according to specific application scenarios and device capabilities. In an actual system, there may be a multi-level sensor switching mechanism. The system will classify, evaluate, and schedule all available sensor resources, and select the most suitable sensor or sensor combination according to the current target manipulation environment, user interaction requirements, and device status.

[0098] Specifically, when the first manipulation environment information does not meet the first manipulation condition, the system can switch from the first manipulation information acquisition sensor to the second manipulation information acquisition sensor.

[0099] Figure 3Schematically shows a scenario diagram of sensor switching provided by an embodiment of the present disclosure.

[0100] As Figure 3 shown, for example, when a user initially uses voice control (the first interaction control method) to operate the vehicle-mounted system 301 during driving, the system mainly relies on the vehicle-mounted microphone 302 (the first control information acquisition sensor) carried by the vehicle-mounted system 301 to collect the user's voice commands. When the vehicle environmental noise level rises sharply and the system determines that the first control environment information does not meet the first control condition. The system determines that the second interaction control method is head pose control, so it switches to the vehicle-mounted camera 303 in the vehicle to capture the driver's head movements, and displays a prompt of "Control mode switched from voice control to head pose control" on the display interface 304 of the vehicle-mounted system.

[0101] It should be noted that even if the user's control information input method remains unchanged, the system may switch different types of sensors for acquisition due to changes in environmental factors. On the contrary, even if the sensor type changes, the user's control information input method may remain unchanged. The sensor switching provided by the embodiments of the present disclosure is independent of the user's control information input method, enabling the system to optimize the information acquisition quality without disturbing the user's habits.

[0102] For example, in the eye movement interaction mode, the system may switch from a visible light eye movement tracking camera to an infrared eye movement tracking camera according to the environmental light conditions. When the user uses eye movement control in a room with moderate light, the visible light eye movement tracking camera can effectively capture eye movements. But when the user enters a strong light environment, the visible light camera may be difficult to accurately identify the pupil position due to the strong light. At this time, the system will automatically switch to the infrared eye movement tracking camera as the second control information acquisition sensor. In the above process, the user always controls the device through eye movement, but the system automatically switches different types of eye movement tracking sensors according to the light conditions.

[0103] By adopting the embodiments of the present disclosure, automatic switching between different types of sensors is achieved. The system can select the most suitable combination of sensor types and interaction methods for different environmental conditions, improving the success rate of controlling electronic devices in complex environments. Secondly, using the most suitable sensor for the current environment and putting other sensors into a low-power state can optimize the energy consumption utilization efficiency of electronic devices.

[0104] To optimize the allocation of sensor resources and the acquisition of control information, on the basis of the above embodiments, as an optional embodiment, during the execution of operation S102, the following operations can also be performed: start the second control information acquisition sensor, and / or, turn off the first control information acquisition sensor.

[0105] Embodiments of the present disclosure provide two sensor application strategies based on different application scenario requirements: a sensor switching strategy and a sensor enhancement strategy. These two strategies are respectively targeted at different usage environments and device conditions, and while ensuring the interaction effect, they achieve reasonable allocation of resources.

[0106] In the sensor switching strategy, when the system determines that the first control environment information does not meet the first control condition, it will first consider completely turning off the first control information acquisition sensor and starting the second control information acquisition sensor.

[0107] This strategy is applicable to resource-constrained electronic devices, such as small wearable devices, which usually face strict power consumption limitations and processing capacity constraints. By only keeping the necessary sensors in working state, the system can significantly reduce energy consumption and optimize the allocation of computing resources.

[0108] For example, in the application scenario of smart glasses, when the user enters a low-light environment from a bright outdoor environment, the system detects that it is difficult for the visible light camera to accurately capture the user's eye movement information. At this time, the system will immediately start the infrared eye movement tracking sensor and turn off the high-power-consuming visible light camera, which not only ensures the accuracy of eye movement tracking but also avoids unnecessary power consumption.

[0109] In the sensor enhancement strategy, the system will start the second control information acquisition sensor and at the same time keep the first control information acquisition sensor working continuously to achieve multi-sensor data fusion.

[0110] This strategy is applicable to scenarios with high requirements for interaction, such as key application scenarios like medical device interaction. By keeping multiple sensors working simultaneously, the system can utilize the complementary characteristics of the sensors to improve the comprehensiveness and accuracy of information acquisition.

[0111] For example, in an intelligent vehicle system, when the driver enters a high-noise environment, resulting in a decrease in the reliability of voice control, the system may start the steering wheel touch sensor as the main control information acquisition sensor, but at the same time keep the microphone array working continuously. In this way, even if the touch operation becomes inconvenient during turning or in an emergency, the system can still capture some of the driver's voice commands, achieve multi-modal interaction, and improve the fault tolerance of the overall system.

[0112] In a feasible implementation, the system can assign different weights and priorities to different sensors. The dominant second control information acquisition sensor obtains a higher processing priority and resource allocation, while the first control information acquisition sensor that remains working serves as an auxiliary input source, performs lightweight processing on the acquired data, and provides supplementary information when necessary.

[0113] It should be noted that the system's allocation and management of sensor resources are not statically fixed, but highly dynamically adaptable. The system continuously adjusts the sensor application strategy according to environmental changes, user usage habits, and device resource status. For example, when the battery power is sufficient, the system may be more inclined to adopt a sensor enhancement strategy to enhance the interaction experience; while when the battery power is low, the system will preferentially adopt a full switching strategy to save energy.

[0114] By adopting the embodiments of the present disclosure, the system can achieve a balance between the device's energy efficiency and interaction performance on the premise of meeting the user's interaction requirements. When adopting the switching strategy, the system can reduce the power consumption of the sensor and extend the device's usage time; while when adopting the sensor enhancement strategy, it can improve the interaction accuracy of the system.

[0115] In related technologies, the interaction system usually regards the collection of environmental information and the collection of user manipulation information as two independent processes, and uses dedicated sensors or collects them at different time periods respectively. This separate information collection method not only increases the system complexity, but also may lead to a time difference between environmental assessment and actual manipulation, and cannot respond in a timely manner to rapidly changing environmental conditions. At the same time, additional environmental information collection sensors will also increase the device cost and power consumption.

[0116] To solve the above problems, on the basis of the above embodiments, as an optional embodiment, the first manipulation environment information in the embodiments of the present disclosure may be directly derived from the information collected by the first manipulation information collection sensor.

[0117] In this case, the first manipulation information collection sensor mainly has two functions. On the one hand, the first manipulation information collection sensor can collect the user's manipulation information for identifying the user's interaction manipulation intention; on the other hand, the first manipulation information collection sensor can simultaneously collect the manipulation environment information for evaluating the manipulation success rate of the first interaction manipulation method.

[0118] Based on this, the above interaction method may further include the following operations:

[0119] Operation S201, obtaining the target data collected by the first manipulation information collection sensor;

[0120] Operation S202, if the first manipulation environment information included in the target data does not meet the first manipulation condition, switching the interaction manipulation method to the second interaction manipulation method;

[0121] Operation S203, if the first manipulation environment information included in the target data meets the first manipulation condition, performing the manipulation operation corresponding to the manipulation information based on the manipulation information included in the target data.

[0122] Specifically, the system continuously collects target data through the first control information acquisition sensor under the current first interaction control method. Among them, the target data refers to the original data set obtained by the system through the first control information acquisition sensor, which can be understood as including control information and control environment information in the embodiments of the present disclosure, and is used for the system to simultaneously evaluate the current environmental suitability and as a basis for identifying the specific control instructions of the user.

[0123] After the system obtains the target data, it first determines whether the first control environment information included in the target data meets the first control condition. If the first control environment information included in the target data does not meet the first control condition, it indicates that the current environment is no longer suitable for continuing to use the first interaction control method, and the system will immediately switch the interaction control method to the second interaction control method. The judgment logic here is the same as that in the above embodiments and will not be elaborated too much here.

[0124] Exemplarily, when the user uses the eye movement tracking function of the smart glasses to control music playback, the integrated eye movement tracking camera will continuously collect the user's eye image sequence as target data. The system analyzes these image data: on the one hand, it extracts environmental parameters such as ambient light intensity and line of sight visibility as the first control environment information; on the other hand, it identifies the user's specific eye movement pattern as the control information.

[0125] When the user walks from indoors to outdoors with strong sunlight, the light intensity parameter extracted by the system from the target data increases significantly and exceeds the threshold applicable to eye movement tracking. At the same time, the system detects that the eye visibility parameter is lower than the preset threshold. At this time, the system determines that the first control environment information included in the target data does not meet the first control condition and switches the eye movement tracking to voice control.

[0126] On the contrary, if the first control environment information included in the target data meets the first control condition, that is, when the user is in an indoor environment suitable for eye movement tracking. At this time, the system determines that the first control environment information meets the first control condition, and then identifies the user's eye movement pattern from the same set of eye image data. The above whole process is completed on the basis of a single data stream without additional environmental sensors or independent environmental assessment processes.

[0127] By adopting the embodiments of the present disclosure, the need for additional environmental perception sensors is reduced, and the device cost and design complexity are lowered; secondly, the environmental assessment and control recognition are based on the data at the same time point, eliminating the time delay between environmental changes and system responses, and improving the system's response speed to environmental changes; in addition, the homology of environmental information and control information improves the consistency of system decisions and avoids errors that may be caused by the out-of-synchronization of different sensor data.

[0128] In related interactive systems, on the one hand, fixed interactive input methods are mostly adopted. However, it is often difficult to adapt to changing environmental conditions, resulting in a significant reduction in the manipulation success rate in specific environments. On the other hand, frequently asking users to switch different input methods will interrupt the operation fluency and increase the user learning cost and cognitive burden. Related technologies either force users to continue using inefficient input methods in unsuitable environments or require users to learn and adapt to completely new input methods every time the environment changes, and cannot achieve the balance between environmental adaptability and user experience.

[0129] To solve the above problems, on the basis of the above embodiments, as an optional embodiment, the first manipulation information input method of the user in the first interactive manipulation method is the same as or different from the second manipulation information input method of the user in the second interactive manipulation method.

[0130] Among them, the manipulation information input method refers to the specific expression form in which the user conveys the manipulation intention to the electronic device through a specific behavior pattern or action. In the embodiments of the present disclosure, it can be understood as the behavior adopted by the user when interacting with the electronic device, and is used to provide the system with manipulable instructions that can be recognized and parsed. The manipulation information input method directly affects the user interaction experience. Different manipulation information input methods have their own characteristics and applicable scenarios. For example, voice input is suitable for situations where the user's hands are restricted, touch input is suitable for scenarios that require precise control, and eye movement input is suitable for environments where the user needs to keep quiet, etc.

[0131] When the first manipulation information input method is the same as the second manipulation information input method, the user uses the same behavior pattern for interaction before and after the switching of the interactive manipulation method. This situation is mainly applicable to scenarios where the user has a high requirement for habit consistency, but environmental factors affect the performance of specific sensors. In this case, the system improves the manipulation success rate while maintaining the consistency of the user experience by switching different types of sensors or adjusting algorithm parameters.

[0132] For example, in the application scenario of smart glasses, the user uses eye movement manipulation as the manipulation information input method (the first manipulation information input method) to control the music player through a specific eye movement pattern. The system uses a camera to capture the user's eye movement in a well-lit indoor environment (the first interactive manipulation method). When the user enters a low-light environment, the capture effect of the camera decreases, and the system detects that the first manipulation environment information does not meet the first manipulation condition. At this time, the system switches the interactive manipulation method to a method based on infrared eye movement tracking technology (the second interactive manipulation method), but the user still uses the same eye movement pattern (the second manipulation information input method is the same as the first manipulation information input method) for control. The user does not need to learn a new interaction method, and the system automatically adapts to environmental changes in the background, providing a seamless interaction experience.

[0133] When the first control information input method is different from the second control information input method, the user needs to change their input behavior before and after switching the interactive control method. This situation mainly applies to scenarios where the original input method is completely ineffective or extremely inefficient in the current environment.

[0134] In this case, the system can also perform the following operations:

[0135] Output a prompt message indicating the switch to the second interactive control method.

[0136] Specifically, the system guides the user to switch to the new input method to ensure effective interaction can still be maintained under complex environmental conditions.

[0137] For example, in the scenario of intelligent vehicle driving, the driver initially uses voice commands (the first control information input method) to control the in-vehicle system, and the system processes the commands through voice recognition technology (the first interactive control method). When the vehicle enters a high-noise environment, the system detects that the first control environment information does not meet the first control conditions. At this time, the system switches the interactive control method to a control method based on the touch buttons on the steering wheel (the second interactive control method), and the driver needs to change the input behavior, from voice commands to pressing the touch buttons on the steering wheel (the second control information input method). Although the input method has changed, the system can ensure that the driver can quickly adapt to the new interaction method through clear prompts and intuitive interface designs, while keeping their attention on the road to ensure driving safety.

[0138] By adopting the embodiments of the present disclosure, the system makes an adaptive judgment based on the severity of the environmental conditions and the importance of user habits, and finds the best balance between environmental adaptability and user experience continuity, thereby improving the overall usability and control success rate of the interactive system in complex and changeable environments.

[0139] Next, the application scenarios of the above embodiments will be further described in combination with application scenarios related to voice interaction.

[0140] In the interactive system of an electronic device, voice interaction is widely used in various scenarios due to its convenience. However, voice interaction highly depends on the quality of audio signals, and problems such as decreased recognition rate, mis-triggering, or inability to trigger are likely to occur in a noisy environment. Users often need to repeatedly try or increase the volume to complete basic operations, seriously affecting the user experience.

[0141] To solve the above problems, based on the above embodiments, as an optional embodiment, the first control environment information includes the environmental audio information of the target control environment, and the first control information input method is to input information through sound;

[0142] The fact that the first control environment information does not meet the first control condition indicates that the audio intensity of the environmental audio in the target control environment is greater than the first audio intensity threshold, and / or the audio intensity of the audio emitted by the target object in the target control environment is less than the second sound intensity threshold.

[0143] Among them, the first audio intensity threshold refers to a preset value for evaluating the environmental noise level, which can be understood as the critical value for judging whether the current environmental noise is too high and not suitable for voice interaction in the embodiments of the present disclosure. Similarly, the second sound intensity threshold refers to a preset value for evaluating the quality of the user voice signal, which can be understood as the minimum intensity requirement for judging whether the voice command issued by the user is clear enough to be effectively recognized by the system in the embodiments of the present disclosure. These two thresholds constitute a dual standard for evaluating the applicability of voice interaction.

[0144] Specifically, the system first continuously collects audio data in the target control environment through the first control information acquisition sensor as the first control environment information. The electronic device can collect environmental sounds through a microphone array and perform preliminary audio signal processing. When the user is in a relatively quiet environment, the microphone array can capture the user's voice command, and the system uses it as the main input of the first interactive control method.

[0145] To further improve the recognition accuracy and environmental adaptability of audio information, as an optional embodiment, the audio information collected by the first control acquisition sensor can also be frequency-divided to obtain the environmental audio of the target control environment and the audio emitted by the target object.

[0146] Specifically, the system performs frequency-division processing on the collected audio data to evaluate the suitability of the current environment for voice interaction. The frequency-division methods include but are not limited to: spectrum analysis, energy calculation, signal-to-noise ratio estimation, etc. After frequency-division processing, the system can extract the environmental audio of the target control environment and the audio emitted by the target object from the overall audio data.

[0147] Furthermore, the system can set scientific and reasonable first audio intensity threshold and second sound intensity threshold according to acoustic principles and a large amount of user test data. The specific value of the first audio intensity threshold can be dynamically adjusted according to the device type, microphone performance, and application scenario to ensure accurate evaluation of the noise impact in different usage environments. Similarly, the second sound intensity threshold can also be adjusted personalized according to the actual user needs and voice characteristics to adapt to the voice characteristics and usage habits of different users.

[0148] Exemplarily, when the system detects that the audio intensity of the ambient noise exceeds the first audio intensity threshold, it indicates that the current ambient noise is too high and may interfere with the recognition of voice commands. For example, when a user is in a noisy subway car or driving on a highway, the ambient noise intensity may reach above 80 dB, far exceeding the first audio intensity threshold. In this case, even if the user raises their speaking volume, the success rate of voice recognition will still decrease significantly, and the user experience will be severely affected.

[0149] Meanwhile, the system will also evaluate whether the audio intensity of the user's voice is lower than the second sound intensity threshold. When the user is unable to issue clear and loud voice commands for various reasons, such as when the user is in a library or meeting room where silence needs to be maintained, or when the user has temporarily lost their voice due to health reasons, the audio intensity of the user's voice may be lower than the second sound intensity threshold preset by the system. In this case, even if the environment is relatively quiet, the success rate of voice interaction will be greatly reduced, unable to meet the user's control requirements.

[0150] When the system determines that the audio intensity of the ambient audio is greater than the first audio intensity threshold, and / or the audio intensity of the audio emitted by the target object is less than the second sound intensity threshold, the system will automatically execute operation S102 to switch the interaction control method from voice control to a second interaction control method more suitable for the current environment. This intelligent switching mechanism based on environmental perception ensures that users can obtain the best interaction experience in any environment.

[0151] When the system selects the second interaction control method, it will comprehensively consider the current environmental characteristics, device type, and user usage habits to select the alternative interaction method most suitable for the current situation. For example, in a high-noise environment, it may switch to touch or gesture control; in an environment where silence needs to be maintained, it may switch to eye movement control.

[0152] By adopting the embodiments of the present disclosure, the system can monitor audio information in real time, actively switch to a more suitable interaction method before the voice interaction effect deteriorates, and avoid the frustrating experience of users repeatedly trying voice commands in a noisy environment. At the same time, the system makes judgments based on clear audio intensity thresholds, making the switching decision more objective and accurate, and reducing the possibility of incorrect switching.

[0153] In addition, the system accurately distinguishes ambient noise and user voice through frequency division processing technology, improving the accuracy and pertinence of environmental assessment. The dual-threshold evaluation further enhances the system's adaptability to complex audio environments, enabling the interaction system to operate efficiently in various acoustic environments.

[0154] In the actual application process, security and privacy protection are also key issues that cannot be ignored. With the diversification of interaction and control methods, security risks such as unauthorized use and accidental triggering have increased, which may lead to information leakage, accidental operation triggering, and other security hazards. Especially in scenarios involving personal privacy or important operations.

[0155] To solve the above problems, based on the above embodiments, as an alternative embodiment, the embodiments of the present disclosure provide an enhanced security and privacy protection mechanism, including authenticating the identity of the target object and designing a security lock to prevent accidental triggering. These two functions complement each other and jointly build a multi-level security protection system.

[0156] Specifically, in the process of implementing the authentication of the identity of the target object, the system can select a suitable authentication method according to the current interaction and control method. When the first interaction and control method is voice control, the system can perform identity verification through voiceprint recognition technology. The electronic device will compare the user's voice characteristics with the pre-stored authorized voiceprint template, and only when the match is successful is the execution of sensitive operations allowed.

[0157] When the first interaction and control method is eye movement tracking, the system can adopt eye movement feature authentication technology. The eye movement pattern of each user is unique, and the system can require the user to complete a specific eye movement sequence as an "eye movement password". For example, the system may require the user to gaze at several different positions on the screen in a preset order, or perform a specific eye movement pattern. Only when the user successfully completes this eye movement verification sequence will the system authorize access to sensitive functions or perform important operations.

[0158] It should be noted that the triggering frequency and intensity of identity authentication will be dynamically adjusted according to the sensitivity of the operation. For general daily operations, the system may only perform identity verification once at the beginning of the interaction session; while for highly sensitive operations (such as payments, access to privacy data), the system may require re-verifying the user's identity before each operation.

[0159] In addition to identity authentication, the embodiments of the present disclosure also set up a security lock mechanism to prevent accidental triggering of interaction and control. Especially in highly sensitive interaction and control methods such as eye movement control, the user's unconscious eye movements may be misinterpreted by the system as control instructions, resulting in accidental operations. The security lock mechanism effectively reduces the accidental triggering rate by introducing specific unlocking conditions.

[0160] Specifically, for eye movement control, the system can set the activation conditions for eye movement interaction. For example, the eye movement control function will only be activated when the user continuously gazes at a specific area for more than a preset time. The system can also set automatic locking conditions. For example, when it detects that the user's attention is distracted (rapid eye movement or frequent blinking) or there is no effective interaction for a long time, the system will automatically lock the eye movement control function to prevent accidental triggering.

[0161] By adopting the embodiments of the present disclosure, while improving the interaction success rate, the security of the interaction system is effectively enhanced. Identity authentication ensures that only authorized users can use the device functions, preventing unauthorized access and operations; the security lock mechanism reduces accidental triggering and avoids the risks brought by inadvertent operations.

[0162] The embodiments of the present disclosure also disclose an electronic device, including:

[0163] A first manipulation information acquisition sensor, configured to acquire manipulation information in the first interaction manipulation mode;

[0164] A second manipulation information acquisition sensor, configured to acquire manipulation information in the second interaction manipulation mode;

[0165] One or more processors, configured to execute:

[0166] When the interaction manipulation mode is the first interaction manipulation mode, obtain the first manipulation environment information corresponding to the first interaction manipulation mode in the target manipulation environment; if the first manipulation environment information does not meet the first manipulation condition, switch the interaction manipulation mode to the second interaction manipulation mode; wherein, the first interaction manipulation mode is different from the second interaction manipulation mode, and in the target manipulation environment, the manipulation success rate based on the first interaction manipulation mode is less than the manipulation success rate based on the second interaction manipulation mode.

[0167] Figure 4 The block diagram of the electronic device provided by the embodiments of the present disclosure is schematically shown. Figure 4 The shown electronic device is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present disclosure.

[0168] Such as Figure 4As shown, an electronic device 400 according to an embodiment of the present disclosure includes a first manipulation information acquisition sensor, a second manipulation information acquisition sensor (not shown in the figure), and a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a memory 408 into a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application-specific integrated circuit (ASIC)), and so on. The processor 401 may also include on-board memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0169] In the RAM 403, various programs and data required for the operation of the electronic device 400 are stored. The processor 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. The processor 401 performs various operations of the method flow according to an embodiment of the present disclosure by executing the programs in the ROM 402 and / or the RAM 403. It should be noted that the programs may also be stored in one or more memories other than the ROM 402 and the RAM 403. The processor 401 may also perform various operations of the method flow according to an embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0170] According to an embodiment of the present disclosure, the electronic device 400 may further include an input / output (I / O) interface 404, and the input / output (I / O) interface 404 is also connected to the bus 404. The system 400 may further include one or more of the following components connected to the input / output (I / O) interface 404: an input device 406 including a keyboard, a mouse, etc.; an output device 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), a display screen, etc., and a speaker, etc.; a memory 408 including a hard disk, etc.; and a communication part 409 including a network interface card such as a LAN card, a modem, etc. The communication part 409 performs communication processing via a network such as the Internet. A driver 410 is also connected to the input / output (I / O) interface 404 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the driver 410 as needed so that a computer program read from it can be installed into the memory 408 as needed.

[0171] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the processor 401, the above functions defined in the system of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0172] The present disclosure also provides a computer-readable storage medium, which may be included in the device / device / system described in the above embodiments; or may exist alone without being assembled into the device / device / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0173] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

[0174] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 402 and / or RAM 403 and / or one or more memories other than ROM 402 and RAM 403.

[0175] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program includes program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to cause the electronic device to implement the method provided by the embodiments of the present disclosure.

[0176] When the computer program is executed by the processor 401, the above functions defined in the system / device of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, devices, modules, units, etc. can be implemented by computer program modules.

[0177] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 409, and / or installed from the removable medium 411. The program code included in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions. Those skilled in the art can understand that the features recited in various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0179] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. An interaction method, comprising: When the interaction control mode is the first interaction control mode, obtaining first control environment information corresponding to the first interaction control mode in a target control environment; If the first control environment information does not meet the first control condition, switching the interaction control mode to a second interaction control mode; Wherein, the first interaction control mode is different from the second interaction control mode, and in the target control environment, the control success rate based on the first interaction control mode is less than the control success rate based on the second interaction control mode.

2. The method according to claim 1, wherein the first control information acquisition sensor for collecting control information in the first interaction control mode is different from the second control information acquisition sensor for collecting control information in the second interaction control mode.

3. The method according to claim 2, wherein the switching the interaction control mode to the second interaction control mode comprises: Starting the second control information acquisition sensor, and / or, closing the first control information acquisition sensor.

4. The method according to claim 2, wherein the first control information input mode of the user in the first interaction control mode is the same as or different from the second control information input mode of the user in the second interaction control mode.

5. The method according to claim 4, wherein the first control environment information comprises environmental audio information of the target control environment, and the first control information input mode is information input by sound; The fact that the first control environment information does not meet the first control condition indicates that the audio intensity of the environmental audio in the target control environment is greater than a first audio intensity threshold, and / or, the audio intensity of the audio emitted by the target object in the target control environment is less than a second sound intensity threshold.

6. The method according to claim 5, further comprising: Dividing the audio information collected by the first control acquisition sensor to obtain the environmental audio of the target control environment and the audio emitted by the target object.

7. The method according to claim 4, wherein the first control information input mode of the user in the first interaction control mode is different from the second control information input mode of the user in the second interaction control mode; The method further comprises: Outputting a prompt message indicating switching to the second interaction control mode.

8. The method according to claim 1, wherein both the first interaction control mode and the second interaction control mode are non-contact interaction control modes.

9. The method according to claim 2, wherein the first control environment information is the information collected by the first control information acquisition sensor; The method further comprises: Obtaining target data collected by the first control information acquisition sensor; If the first control environment information included in the target data does not meet the first control condition, switching the interaction control mode to the second interaction control mode; If the first control environment information included in the target data meets the first control condition, performing a control operation corresponding to the control information based on the control information included in the target data.

10. An electronic device, comprising: The first manipulation information acquisition sensor is used to acquire manipulation information under the first interaction manipulation mode; The second manipulation information acquisition sensor is used to acquire manipulation information under the second interaction manipulation mode; One or more processors are used to execute: When the interaction manipulation mode is the first interaction manipulation mode, obtain the first manipulation environment information corresponding to the first interaction manipulation mode in the target manipulation environment; if the first manipulation environment information does not meet the first manipulation condition, switch the interaction manipulation mode to the second interaction manipulation mode; wherein, the first interaction manipulation mode is different from the second interaction manipulation mode, and in the target manipulation environment, the manipulation success rate based on the first interaction manipulation mode is less than the manipulation success rate based on the second interaction manipulation mode.

Images