Notebook computer control method and system
By combining multimodal sensors with lightweight neural networks, the human-computer interaction of laptops is dynamically adjusted, solving the problems of inaccurate scenario judgment and insufficient adaptability in existing technologies, and realizing intelligent and personalized user experience and multi-device collaborative support.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-17
AI Technical Summary
Existing laptop control methods and systems lack multimodal information fusion perception, resulting in one-sided and inaccurate situation judgment, adaptive adjustment limited to a few parameters, failure to deeply optimize touch logic, lack of user feedback error correction mechanism, and difficulty in adapting to multi-device collaborative scenarios.
By collecting user status and environmental data through multimodal sensors and combining it with a lightweight neural network model for real-time analysis, the human-computer interaction module is dynamically adjusted, an error correction feedback mechanism is introduced, and collaborative device perception and cross-device interaction are supported.
It achieves real-time, intelligent user scenario recognition and personalized interaction optimization, improves the smoothness and naturalness of operation, has self-learning capabilities, and adapts to modern multi-device collaborative scenarios.
Smart Images
Figure CN121879566A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of human-computer interaction and computer control technology, and more specifically, to a laptop computer control method and system. Background Technology
[0002] With the rapid development of computer technology, especially mobile computing devices, laptops have become core tools for work, study, and entertainment. Users have higher demands for the interactive experience of laptops, hoping that the devices can more intelligently understand the usage scenario and automatically adjust their behavior to adapt to different task requirements, such as precise office work, immersive reading, meeting presentations, or leisure and entertainment. Traditional laptops mostly use static settings for human-computer interaction, requiring users to manually switch modes or adjust parameters. They cannot dynamically adapt to the real-time usage environment and user status, which to some extent affects operating efficiency and user experience.
[0003] However, existing laptop control methods and systems still have significant drawbacks: First, most solutions rely solely on simple contextual information from a single type of sensor or application (such as switching modes based solely on application type or ambient light), lacking the fusion perception and comprehensive analysis of multimodal information such as user eye movements, posture, and ambient acoustics. This results in biased and inaccurate scenario judgments, easily leading to false triggers or delayed responses. Second, adaptive adjustment strategies are usually limited to a few parameters such as screen brightness and volume, failing to delve into dynamic optimization at core interaction levels such as touch logic, input device mapping, and function key combinations, resulting in low personalization and scenario fit. Third, these systems generally lack continuous learning and user feedback error correction mechanisms, making it impossible to self-correct once a misjudgment occurs, and it is also difficult to accumulate user preferences to achieve increasingly accurate personalized adaptation over time. Fourth, existing methods rarely consider the status perception and cross-device interaction integration of surrounding collaborative devices (such as mobile phones, tablets, and external monitors), making it difficult to support the increasingly common multi-device collaborative office and entertainment scenarios.
[0004] Based on this, the present invention designs a laptop computer control method and system to solve the above problems. Summary of the Invention
[0005] The purpose of this invention is to provide a laptop computer control method and system to solve the problems mentioned in the background art.
[0006] A method for controlling a laptop computer includes the following steps: S1. Multimodal environment perception: Real-time collection of user status data and environmental status data through the built-in sensor group of the laptop. The user status data and environmental status data are used to comprehensively infer the user's operating scenario and interaction intention. The user status data includes user eye movement information, facial orientation and posture image. The environmental status data includes ambient light intensity and background noise level. S2. Contextual Analysis: The user state data and environmental state data collected in S1 are jointly analyzed with the type of application currently running in the foreground. The results are then input into a pre-trained lightweight neural network model for real-time inference. The pre-trained lightweight neural network model analyzes the user state data to assess the user's focus and interaction posture, analyzes the environmental state data to assess the level of environmental interference, and combines the application type to comprehensively determine the current user control scenario. The user control scenario includes precision operation mode, immersive reading mode, demonstration and explanation mode, and relaxation and entertainment mode. S3. Adaptive control strategy execution: Based on the user control scenario determined in step S2, dynamically adjust the control logic and parameters of at least one core human-computer interaction module. The dynamic adjustment includes: automatically switching the preferred input device according to the user control scenario, adjusting the acceleration curve and sensitivity of the touchpad cursor movement, remapping the combination logic of specific function keys, and adjusting the microphone noise reduction level and screen color temperature.
[0007] Preferably, in step S2, the real-time analysis and determination of the current user control scenario using a pre-trained lightweight neural network model specifically includes: S2.1. Extract and fuse features from sensor data to generate multidimensional feature vectors; S2.2 Input the multidimensional feature vector and the current application identifier into the pre-trained lightweight neural network model for inference; S2.3 The pre-trained lightweight neural network model outputs a decision based on the probability distribution of each preset scenario, and determines the scenario with the highest probability as the current user control scenario.
[0008] Preferably, the dynamic adjustment of the control logic and parameters of at least one core human-computer interaction module in step S3 further includes: S3.1 When the scene is switched to immersive reading mode, scroll control based on camera eye tracking is automatically enabled, and the touchpad area is temporarily mapped as a page flipping area; S3.2 When the scenario switches to the presentation mode, the touch screen or touchpad edge sliding area is automatically mapped to the presentation page turning shortcut key, and the click feedback intensity of the pointing device is enhanced.
[0009] Preferably, after step S3, step S4 is also included: S4, Error Correction Feedback and Model Iteration, specifically includes: S4.1 After executing the adaptive control strategy, continuously monitor the preset user denial behavior, which includes triggering a preset shortcut key combination that is opposite to the recommended operation in the current scenario under a specific scenario, or repeatedly executing the undo operation within a short period of time. S4.2 When the user's negative behavior is detected, immediately provide a switching menu containing at least two alternative scenarios for the user to choose from, and restore the scenario control strategy after the user's selection; S4.3. Use the data sequence of the incorrect judgment and the correct scenario finally selected by the user as feedback data to incrementally learn the lightweight neural network model in step S2 and optimize the accuracy of subsequent scenario analysis.
[0010] Preferably, the real-time parsing and determination of the current user control scenario in step S2 further includes: S2.4. Detect and identify collaborative devices and their types that are in operation around the user through a wireless connection protocol. The collaborative devices include at least smartphones, tablets, and smartwatches. S2.5. The type and status of the collaborative device are used as additional input features to participate in the model reasoning in step S2.2 or the scenario decision-making in step S2.3. Specifically, step S3, which involves dynamically adjusting the control logic and parameters of at least one core human-computer interaction module, further includes: S3.3 When the current scenario is identified as a multi-tasking office mode and an external monitor is present, a specific area of the touchpad or touch screen is automatically mapped as a shortcut area for cross-monitor window layout management. The sliding operation can directly send the current window to the specified location on the external monitor.
[0011] Preferably, the pre-trained lightweight neural network model is a recurrent neural network or a temporal convolutional network with temporal memory capability; The feature extraction and fusion in step S2.1 includes: combining the multidimensional feature vector with the feature vectors of historical moments to form a temporal feature sequence; Step S2.3 specifically involves the pre-trained lightweight neural network model outputting a prediction of the stability of user intent over a future period based on the temporal feature sequence. Combined with the probability distribution at the current moment, scenario switching is only performed when the predicted stability is higher than the threshold and the probability distribution shows a clear main peak. Otherwise, the previous stable scenario is maintained or a general, highly compatible basic control mode is entered.
[0012] A laptop computer control system includes: Multimodal environment perception module: integrates heterogeneous cameras, ambient light sensors, microphone arrays and inertial measurement units to collect user status data and environmental status data in real time; Contextual intelligent analysis module: connected to the multimodal environment perception module and the system application management layer, with a built-in pre-trained lightweight neural network model, used to determine the user control context in real time based on perception data and application information; Adaptive control execution engine: Connected to the scenario intelligent analysis module, the adaptive control execution engine sends instructions to the human-computer interaction subsystem of the operating system according to the determined scenario, and dynamically configures the operating parameters of the touchpad driver, keyboard controller, display driver and audio driver.
[0013] Preferably, the scenario intelligent analysis module includes: The feature fusion unit is used to perform time alignment, standardization and feature fusion on the heterogeneous sensor data from the multimodal environment perception module to generate a multidimensional feature vector that represents the current comprehensive state of the user and the environment. The model inference unit loads the pre-trained lightweight neural network model, receives the multi-dimensional feature vector and application context signals from the system application management layer, and performs forward inference calculations. The scenario decision-making unit, based on the probability distribution output by the pre-trained lightweight neural network model and combined with the user's historical preference database, performs fine-tuning and finally outputs the determined scenario identifier to the adaptive control execution engine.
[0014] Preferably, the adaptive control execution engine includes: The policy configuration library stores device parameter configuration templates and logical mapping rules that correspond one-to-one with different user control scenarios. The dynamic loader, in response to the scenario switching command, loads the corresponding configuration template and rules from the policy configuration library; The device interface adaptation layer converts the loaded configuration templates and rules into specific call instructions that can be executed by the operating system's underlying input / output device drivers, thus achieving seamless switching of control strategies.
[0015] Compared with the prior art, the advantages of this invention are: 1. This invention combines multimodal environment perception with contextual scenario analysis, enabling real-time and intelligent identification of user operation scenarios and intentions. This achieves a shift from passive response to proactive adaptation in human-computer interaction, significantly improving the smoothness and naturalness of the user experience.
[0016] 2. This invention uses a pre-trained lightweight neural network model for real-time reasoning and decision-making. While ensuring the accuracy of scene recognition, it features low latency and low resource consumption, making it suitable for the real-time control needs of mobile computing devices such as laptops.
[0017] 3. This invention has the ability to dynamically adjust adaptive control strategies, and can automatically optimize multi-dimensional interactive settings such as input device mapping, touch response, and audio and video parameters according to different scenarios, so as to achieve personalized interactive optimization with one strategy for one scenario.
[0018] 4. This invention introduces error correction feedback and model iteration mechanisms, which can correct misjudgments in real time through user behavior and continuously optimize the neural network model, enabling the system to have the ability to learn and evolve continuously, becoming smarter the more it is used.
[0019] 5. This invention supports extended scenarios such as collaborative device sensing and external display management, and can identify and integrate the collaborative working status of multiple user devices, achieving seamless interaction and integration across devices and screens, and adapting to the complex needs of modern mobile office and entertainment. Attached Figure Description
[0020] Figure 1 This is an architecture diagram of the modules of a laptop computer control method and system proposed in this invention; Figure 2 This is a flowchart of a laptop computer control method and system proposed in this invention. Detailed Implementation
[0021] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] Please see Figures 1-2 A method for controlling a laptop computer includes the following steps: S1. Multimodal environment perception: Real-time collection of user status data and environmental status data through the built-in sensor group of the laptop. The user status data and environmental status data are used to comprehensively infer the user's operating scenario and interaction intention. Among them, the user status data includes user eye movement information, facial orientation and posture image, and the environmental status data includes ambient light intensity and background noise level. The sensor array consists of at least an infrared camera (for eye and face tracking), an RGB camera (for posture recognition), an ambient light sensor, a three-axis accelerometer and gyroscope (for device posture perception), and a microphone array (for sound source localization and noise analysis). Processing of user status data: Eye movement information is extracted by pupil center and corneal reflection method to extract the gaze point; facial orientation is calculated by feature point detection and head posture estimation model; posture image is identified by lightweight human key point detection network to identify user sitting posture, arm position, etc. Environmental status data processing: Ambient light intensity is sampled in real time by a light sensor and smoothed and filtered; background noise level is collected by a microphone array, and the signal-to-noise ratio and steady-state noise energy are calculated after frequency domain analysis. S2. Contextual Analysis: The user state data and environmental state data collected in S1 are jointly analyzed with the type of application currently running in the foreground. The data is then input into a pre-trained lightweight neural network model for real-time inference. The pre-trained lightweight neural network model analyzes user state data to assess user focus and interaction posture, analyzes environmental state data to assess the level of environmental interference, and combines the application type to comprehensively determine the current user control scenario. User control scenarios include precision operation mode, immersive reading mode, demonstration and explanation mode, and relaxation and entertainment mode. The pre-trained lightweight neural network model is obtained through the following training method: First, a labeled training set covering multiple scenarios is constructed, where each sample includes multimodal sensor data after time alignment and enhancement processing, front-end application identifiers, and corresponding user control scenario labels; then, a lightweight network architecture such as MobileNetV3 is selected, and the dataset is trained in a supervised manner using the cross-entropy loss function and the Adam optimizer, so that the model learns to identify different scenarios from fused features; after training, the model is pruned and quantized to adapt to edge deployment; during system operation, misjudged samples are collected through the error correction feedback mechanism of claim 4, and the model is fine-tuned periodically in an incremental learning manner, thereby achieving continuous optimization and personalized adaptation; S3. Adaptive control strategy execution: Based on the user control scenario determined in step S2, dynamically adjust the control logic and parameters of at least one core human-computer interaction module. The dynamic adjustment includes: automatically switching the preferred input device according to the user control scenario, adjusting the acceleration curve and sensitivity of the touchpad cursor movement, remapping the combination logic of specific function keys, and adjusting the microphone noise reduction level and screen color temperature.
[0023] In step S2, the current user control scenario is analyzed and determined in real time using a pre-trained lightweight neural network model, specifically including: S2.1. Extract and fuse features from sensor data to generate multidimensional feature vectors; S2.2 Input the multi-dimensional feature vector and the current application identifier into the pre-trained lightweight neural network model for inference; S2.3 The pre-trained lightweight neural network model outputs a decision based on the probability distribution of each preset scenario, and determines the scenario with the highest probability as the current user control scenario.
[0024] Feature extraction and fusion methods: Convolutional neural networks (CNNs) are used to extract image features, time-frequency transformation is used to extract audio features, statistical methods are used to extract sensor time-series features, and then fully connected layers are used to concatenate and reduce the dimensions of features to generate a unified multidimensional feature vector.
[0025] Lightweight neural network architecture: The model is a variant of MobileNetV3 or SqueezeNet, with fewer than 2M parameters, supporting real-time inference on the CPU (latency <50ms).
[0026] Probability distribution decision mechanism: Softmax is used to output the probability of each scenario, and a probability threshold (e.g., >0.6) is set to trigger scenario switching, thus avoiding frequent false switching.
[0027] Step S3, which involves dynamically adjusting the control logic and parameters of at least one core human-computer interaction module, also includes: S3.1 When the scene is switched to immersive reading mode, scroll control based on camera eye tracking is automatically enabled, and the touchpad area is temporarily mapped as a page flipping area; S3.2 When the scenario switches to the presentation mode, the touch screen or touchpad edge sliding area is automatically mapped to the presentation page turning shortcut key, and the click feedback intensity of the pointing device is enhanced.
[0028] Eye-tracking scrolling control: In immersive reading mode, the camera is enabled to continuously track the user's gaze point. When the gaze point approaches the bottom of the page, the page is automatically and smoothly scrolled. The scrolling speed can be adaptively adjusted according to the gaze duration.
[0029] Touchpad area remapping: The bottom left and bottom right corners of the touchpad are mapped as the previous and next page quick flipping areas, respectively, and the page flipping range can be adjusted by swiping gestures.
[0030] Demonstration of page turning shortcut key mapping: In demonstration and explanation mode, the left and right edges of the touchpad are mapped to the previous page and the next page respectively, and the vibration motor simulates the click feedback of physical buttons; Once recognition is successful, the computer will automatically switch to the state most suitable for the scenario: for example, the screen becomes softer and supports eye-tracking page turning when reading; microphone noise reduction is enhanced and the touchpad edge is set as page turning keys when giving a presentation; and sound effects and screen colors are optimized when relaxing and entertaining.
[0031] Following step S3, step S4 is also included: S4, Error Correction Feedback and Model Iteration, specifically includes: S4.1 After executing the adaptive control strategy, continuously monitor the preset user negative behaviors. User negative behaviors include triggering a preset shortcut key combination that is opposite to the recommended operation in the current scenario under specific circumstances, or repeatedly executing the undo operation within a short period of time. S4.2 When a user's negative behavior is detected, immediately provide a switching menu containing at least two alternative scenarios for the user to choose from, and restore the scenario control strategy after the user's selection; S4.3. Use the data sequence of the incorrect judgment and the correct scenario finally selected by the user as feedback data to incrementally learn the lightweight neural network model in step S2 and optimize the accuracy of subsequent scenario analysis.
[0032] The specific definition of user negative behavior includes, but is not limited to: pressing Ctrl+Z (undo) within 2 seconds after the system automatically enters immersive reading mode; clicking the touchpad three times in quick succession in presentation mode (considered as accidental touch); or actively lowering the volume to mute in relaxation and entertainment mode (considered as environmental misjudgment).
[0033] The interactive design of the switching menu is as follows: it pops up in the form of a semi-transparent floating panel, displaying 2-4 of the most likely scenario options, and supports quick selection by voice or keyboard shortcuts.
[0034] Incremental learning mechanism: The training samples are composed of erroneous data sequences and the correct scenarios selected by the user. The last layer of the neural network is updated in an online learning manner, and the entire model is fine-tuned once a week.
[0035] Step S2, which involves real-time analysis and determination of the current user control scenario, also includes: S2.4. Detect and identify collaborative devices and their types that are in operation around the user through a wireless connection protocol. Collaborative devices include at least smartphones, tablets, and smartwatches. S2.5. The type and status of the collaborative devices are used as additional input features to participate in the model reasoning in step S2.2 or the scenario decision-making in step S2.3. Specifically, step S3, which involves dynamically adjusting the control logic and parameters of at least one core human-computer interaction module, further includes: S3.3 When the current scenario is identified as a multi-tasking office mode and an external monitor is present, a specific area of the touchpad or touch screen is automatically mapped as a shortcut area for cross-monitor window layout management. The sliding operation can directly send the current window to the specified location on the external monitor.
[0036] Collaborative device detection protocol: Supports Bluetooth 5.0, Wi-Fi Direct and NFC near field communication to identify device type (such as mobile phone, tablet, watch) and currently active application (such as mobile phone is making a call, tablet is drawing).
[0037] External monitor management in multi-tasking office mode: When the system recognizes an external monitor and reads its resolution through EDID, it divides the touchpad into four areas, corresponding to the left half of the main screen, the right half of the main screen, the left half of the extended screen, and the right half of the extended screen, respectively. It supports three-finger swipe to quickly move the current window to the target area.
[0038] The pre-trained lightweight neural network model is a recurrent neural network or a temporal convolutional network with temporal memory capabilities; The feature extraction and fusion in step S2.1 includes: combining the multidimensional feature vector with the feature vectors of historical moments to form a temporal feature sequence; Step S2.3 is as follows: The pre-trained lightweight neural network model outputs a prediction of the stability of user intent over a future period based on the temporal feature sequence. Combined with the probability distribution at the current moment, the scenario switch is only executed when the predicted stability is higher than the threshold and the probability distribution shows a clear main peak. Otherwise, the previous stable scenario is maintained or the general high-compatibility basic control mode is entered.
[0039] Temporal memory network structure: Using GRU or TCN model, the input is the feature sequence of the most recent 10 seconds, and the output is the user intent stability score (0-1) for the next 5 seconds.
[0040] Stability threshold setting: The default threshold is 0.7, which users can adjust in the settings; if the stability is lower than the threshold, the system will maintain the current mode or switch to general mode, in which all interaction parameters are set to neutral values to avoid interfering with the user.
[0041] A laptop computer control system includes: Multimodal environment perception module: integrates heterogeneous cameras, ambient light sensors, microphone arrays and inertial measurement units to collect user status data and environmental status data in real time; Contextual intelligent analysis module: Connects to the multimodal environment perception module and the system application management layer, and has a built-in pre-trained lightweight neural network model to determine the user control context in real time based on perception data and application information; Adaptive Control Execution Engine: Connected to the Context Intelligence Analysis Module, the Adaptive Control Execution Engine sends instructions to the human-computer interaction subsystem of the operating system based on the determined context, and dynamically configures the operating parameters of the touchpad driver, keyboard controller, display driver and audio driver.
[0042] The contextual intelligent analysis module includes: The feature fusion unit is used to perform time alignment, standardization and feature fusion on heterogeneous sensor data from the multimodal environment perception module to generate a multidimensional feature vector that represents the current integrated state of the user and the environment. The model inference unit loads a pre-trained lightweight neural network model, receives multi-dimensional feature vectors and application context signals from the system application management layer, and performs forward inference computation. The scenario decision-making unit fine-tunes the probability distribution output by the pre-trained lightweight neural network model in conjunction with the user's historical preference database, and finally outputs the determined scenario identifier to the adaptive control execution engine.
[0043] The adaptive control execution engine includes: The policy configuration library stores device parameter configuration templates and logical mapping rules that correspond one-to-one with different user control scenarios. The dynamic loader, in response to scenario switching commands, loads the corresponding configuration templates and rules from the policy configuration library; The device interface adaptation layer converts the loaded configuration templates and rules into specific call instructions that can be executed by the operating system's underlying input / output device drivers, thus achieving seamless switching of control strategies.
[0044] Hardware module integration method: Sensor data is accessed via I²C / SPI bus to embedded coprocessor (such as ARM Cortex-M4), preprocessed, and then transmitted to the main system via USB or PCIe.
[0045] Deployment of the contextual intelligent analysis module: It resides in memory as a system service, with a priority set higher than ordinary applications but lower than kernel drivers to ensure real-time performance.
[0046] The instruction passing path of the adaptive control execution engine: It can directly modify device driver parameters through the HID API, display management interface and audio policy manager provided by the operating system, without restarting the driver or application.
[0047] The workflow of this invention is as follows: I. Multimodal Environmental Perception and Data Fusion The system uses a heterogeneous sensor array built into the laptop, including infrared and RGB cameras, an ambient light sensor, a microphone array, and an inertial measurement unit, to simultaneously collect raw data such as user eye movement, facial orientation, body posture, ambient light, and background noise. Subsequently, the raw data is time-aligned, filtered, and standardized, and feature extraction algorithms, such as CNN to extract visual features and time-frequency analysis to extract audio features, are used to fuse these multimodal data into a unified multidimensional feature vector that represents the current comprehensive state of the user and the environment.
[0048] II. Contextual Context Analysis and Decision Making The system combines the multi-dimensional feature vectors generated in the first stage with the type of application currently running in the foreground and contextual information such as the status of cooperating devices detected through wireless protocols, such as mobile phones, tablets, etc., and inputs them into a pre-trained lightweight neural network model, such as a temporal convolutional network (TCN). This model performs real-time inference and outputs the probability distribution of various preset scenarios that the user may be in, such as precise operation, immersive reading, demonstration and explanation, etc. The model combines historical time-series information to predict the stability of the user's intention. Only when the stability is high and the probability of a certain scenario is significantly prominent will the scenario identifier be finally determined and output, thereby ensuring the accuracy and consistency of decision-making.
[0049] III. Adaptive Control Strategy Execution and Switching Based on the scenario identifiers output in the second phase, the system immediately retrieves the corresponding device parameter templates and logical mapping rules from the pre-set strategy configuration library. The adaptive control execution engine then dynamically adjusts the operating parameters of the core human-computer interaction module through the operating system interface. For example, in immersive reading mode, eye-tracking page turning is enabled and the touchpad is mapped as a quick page-turning area; in demonstration and explanation mode, the touchpad edge is mapped as page-turning keys and click feedback is enhanced; and when an external monitor is detected, the cross-screen window management shortcut area is automatically enabled, achieving seamless and precise switching of control strategies.
[0050] IV. Continuous monitoring, error correction feedback, and model iteration After the control strategy is executed, the system continuously monitors whether the user exhibits negative behavior, such as performing an operation opposite to the current scenario recommendation. Once a misjudgment is detected, the system immediately provides an alternative scenario menu for the user to manually correct. The original data sequence of this misjudgment, along with the correct scenario ultimately selected by the user, will serve as high-quality feedback data for incremental learning of the neural network model in the second stage. Through this continuous online learning mechanism, the system can continuously optimize the accuracy of its scenario analysis, achieving a personalized adaptability that becomes increasingly intelligent with use.
[0051] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for controlling a laptop computer, characterized in that, Includes the following steps: S1. Multimodal environment perception: Real-time collection of user status data and environmental status data through the built-in sensor group of the laptop. The user status data and environmental status data are used to comprehensively infer the user's operating scenario and interaction intention. The user status data includes user eye movement information, facial orientation and posture image. The environmental status data includes ambient light intensity and background noise level. S2. Contextual Analysis: The user state data and environmental state data collected in S1 are jointly analyzed with the type of application currently running in the foreground. The results are then input into a pre-trained lightweight neural network model for real-time inference. The pre-trained lightweight neural network model analyzes the user state data to assess the user's focus and interaction posture, analyzes the environmental state data to assess the level of environmental interference, and combines the application type to comprehensively determine the current user control scenario. The user control scenario includes precision operation mode, immersive reading mode, demonstration and explanation mode, and relaxation and entertainment mode. S3. Adaptive control strategy execution: Based on the user control scenario determined in step S2, dynamically adjust the control logic and parameters of at least one core human-computer interaction module. The dynamic adjustment includes: automatically switching the preferred input device according to the user control scenario, adjusting the acceleration curve and sensitivity of the touchpad cursor movement, remapping the combination logic of specific function keys, and adjusting the microphone noise reduction level and screen color temperature.
2. The notebook computer control method according to claim 1, characterized in that, In step S2, the real-time analysis and determination of the current user control scenario using a pre-trained lightweight neural network model specifically includes: S2.
1. Extract and fuse features from sensor data to generate multidimensional feature vectors; S2.2 Input the multidimensional feature vector and the current application identifier into the pre-trained lightweight neural network model for inference; S2.3 The pre-trained lightweight neural network model outputs a decision based on the probability distribution of each preset scenario, and determines the scenario with the highest probability as the current user control scenario.
3. The laptop computer control method according to claim 1, characterized in that, The dynamic adjustment of the control logic and parameters of at least one core human-computer interaction module in step S3 also includes: S3.1 When the scene is switched to immersive reading mode, scrolling control based on camera eye tracking is automatically enabled, and the touchpad area is temporarily mapped as a page flipping area; S3.2 When the scenario switches to the presentation mode, the touch screen or touchpad edge sliding area is automatically mapped to the presentation page turning shortcut key, and the click feedback intensity of the pointing device is enhanced.
4. The notebook computer control method according to claim 1, characterized in that, Following step S3, step S4 is also included: S4, Error Correction Feedback and Model Iteration, specifically includes: S4.1 After executing the adaptive control strategy, continuously monitor the preset user denial behavior, which includes triggering a preset shortcut key combination that is opposite to the recommended operation in the current scenario under a specific scenario, or repeatedly executing the undo operation within a short period of time; S4.2 When the user's negative behavior is detected, immediately provide a switching menu containing at least two alternative scenarios for the user to choose from, and restore the scenario control strategy after the user's selection; S4.
3. Use the data sequence of the incorrect judgment and the correct scenario finally selected by the user as feedback data to incrementally learn the lightweight neural network model in step S2 and optimize the accuracy of subsequent scenario analysis.
5. A laptop computer control method according to claim 4, characterized in that, The real-time parsing and determination of the current user control scenario in step S2 also includes: S2.
4. Detect and identify collaborative devices and their types that are in operation around the user through a wireless connection protocol. The collaborative devices include at least smartphones, tablets, and smartwatches. S2.
5. The type and status of the collaborative device are used as additional input features to participate in the model reasoning in step S2.2 or the scenario decision-making in step S2.
3. Specifically, step S3, which involves dynamically adjusting the control logic and parameters of at least one core human-computer interaction module, further includes: S3.3 When the current scenario is identified as a multi-tasking office mode and an external monitor is present, a specific area of the touchpad or touch screen is automatically mapped as a cross-monitor window layout management shortcut area. The sliding operation can directly send the current window to the specified location on the external monitor.
6. A laptop computer control method according to claim 2, characterized in that, The pre-trained lightweight neural network model is a recurrent neural network or a temporal convolutional network with temporal memory capability; The feature extraction and fusion in step S2.1 includes: combining the multidimensional feature vector with the feature vectors of historical moments to form a temporal feature sequence; Step S2.3 specifically involves the pre-trained lightweight neural network model outputting a prediction of the stability of user intent over a future period based on the temporal feature sequence. Combined with the probability distribution at the current moment, scenario switching is only performed when the predicted stability is higher than the threshold and the probability distribution shows a clear main peak. Otherwise, the previous stable scenario is maintained or a general, highly compatible basic control mode is entered.
7. A laptop computer control system, characterized in that, include: Multimodal environmental perception module: integrates heterogeneous cameras, ambient light sensors, microphone arrays and inertial measurement units to collect user status data and environmental status data in real time; Contextual intelligent analysis module: connected to the multimodal environment perception module and the system application management layer, with a built-in pre-trained lightweight neural network model, used to determine the user control context in real time based on perception data and application information; Adaptive control execution engine: Connected to the scenario intelligent analysis module, the adaptive control execution engine sends instructions to the human-computer interaction subsystem of the operating system according to the determined scenario, and dynamically configures the operating parameters of the touchpad driver, keyboard controller, display driver and audio driver.
8. A notebook computer control system according to claim 7, characterized in that, The scenario intelligent analysis module includes: The feature fusion unit is used to perform time alignment, standardization and feature fusion on the heterogeneous sensor data from the multimodal environment perception module to generate a multidimensional feature vector that represents the current comprehensive state of the user and the environment. The model inference unit loads the pre-trained lightweight neural network model, receives the multi-dimensional feature vector and application context signals from the system application management layer, and performs forward inference calculations. The scenario decision-making unit, based on the probability distribution output by the pre-trained lightweight neural network model and combined with the user's historical preference database, performs fine-tuning and finally outputs the determined scenario identifier to the adaptive control execution engine.
9. A laptop computer control system according to claim 7, characterized in that, The adaptive control execution engine includes: The policy configuration library stores device parameter configuration templates and logical mapping rules that correspond one-to-one with different user control scenarios. The dynamic loader, in response to the scenario switching command, loads the corresponding configuration template and rules from the policy configuration library; The device interface adaptation layer converts the loaded configuration templates and rules into specific call instructions that can be executed by the operating system's underlying input / output device drivers, thus achieving seamless switching of control strategies.