Intelligent interaction state tracking method and system based on human-computer collaboration

Abstract

The application discloses a human-computer collaborative intelligent interaction state tracking method and system, relates to the technical field of human-computer collaborative interaction, and comprises the following steps: acquiring first feature data and second feature data; extracting amplitude information of the first feature data, and generating a first identifier and a first parameter according to whether the amplitude information is lower than a first threshold value; in the case that the first identifier is captured, third feature data of the second feature data is extracted; based on a second identifier of a terminal, the first parameter and the third feature data, target score information is determined; in the case that the target score information satisfies a second threshold value, a target signal is issued to an external collaborative device, and an asynchronous state backtracking check is triggered; the application avoids the emergency interaction instruction of the cloud traditional Chinese medicine platform being ignored in an extreme care scene.

Description

Technical Field

[0001] This invention relates to the field of human-computer collaborative interaction technology, specifically to a method and system for tracking intelligent interaction status based on human-computer collaboration. Background Technology

[0002] With the increasing application of smart cloud TCM platforms in the prevention and management of chronic diseases (such as hypertension, diabetes and complications), the system typically relies on IoT wearable edge terminals to monitor patients' health and interaction data in real time, so that cross-expert physicians can carry out precise personalized diagnosis and treatment and rehabilitation intervention.

[0003] However, in remote medical and elderly care scenarios that are separate from physical TCM clinics, if a patient suddenly suffers an acute illness that leads to motor disability, their urgent SOS calls to the cloud platform are often manifested as extremely weak and short tremors due to physiological limitations. At this time, the fixed threshold anti-accidental touch filtering mechanism configured by traditional monitoring systems to shield against daily physical disturbances will directly judge these emergency interaction intentions as invalid noise and block them. This rigid static defense logic causes the edge terminal to fail to respond during the critical window when the patient most needs the platform's intervention, blocking the collaborative communication link between the terminal and the platform's shared service resources such as expert doctors and pharmacies, ultimately resulting in the neglect of highly risky emergency interaction commands. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for intelligent interactive state tracking based on human-machine collaboration.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for tracking the state of intelligent interaction based on human-machine collaboration, comprising: Obtain the first feature data and the second feature data; Extract the amplitude information of the first feature data, and generate a first identifier and a first parameter based on whether the amplitude information is lower than a first threshold; If the first identifier is captured, the third feature data of the second feature data is extracted; Based on the terminal's second identifier, the first parameter, and the third feature data, the target scoring information is determined. If the target score information meets the second threshold, a target signal is sent to an external collaborative device, and asynchronous state backtracking verification is triggered.

[0006] As a preferred embodiment of the present invention, determining the target scoring information includes: The current working range of the terminal is determined based on the second identifier; When the working interval is the first interval, weight information is determined based on the first parameter, and the second feature data is weighted and fused based on the weight information to obtain the target score information; When the working interval is the second interval, the fourth feature data is extracted from the second feature data, and the target score information is obtained based on the comparison result of the fourth feature data and the physiological feature template.

[0007] As a preferred embodiment of the present invention, the step of determining the weight information based on the first parameter includes: When the third feature data meets the third threshold, the first parameter is input into a preset nonlinear exponential amplification function to calculate the physiological weight amplification coefficient, which is used as the weight information. The value of the first parameter is positively correlated with the physiological weight amplification coefficient.

[0008] As a preferred embodiment of the present invention, the extraction of the fourth feature data from the second feature data includes: Perform a Fast Fourier Transform on the second feature data to extract a subset of features within a preset frequency domain range as the fourth feature data; The second range is determined by the power percentage or hardware load information indicated by the second identifier.

[0009] As a preferred embodiment of the present invention, before determining the target scoring information, the method further includes: Calculate the modality pointing information corresponding to the first feature data and the third feature data respectively; If the deviation of the modality pointing information in different modalities exceeds a preset conflict threshold, a conflict penalty factor is introduced to correct the target scoring information.

[0010] As a preferred embodiment of the present invention, the triggering of asynchronous state backtracking verification includes: Cache the combined sequence data of the first feature data and the second feature data within a preset time window; The joint sequence data is subjected to trend fitting processing, and the target signal is determined as a false triggering signal based on the processing result. If the target signal is the false trigger signal, a second instruction is generated and sent to the external collaborative device to cancel the collaborative state corresponding to the target signal.

[0011] As a preferred embodiment of the present invention, after acquiring the first feature data and the second feature data, the method further includes: If the second feature data is continuously empty within a preset period, a third identifier is generated; Based on the distribution frequency of the third identifier and the first parameter, the terminal is triggered to issue an audible and visual alarm.

[0012] As a preferred embodiment of the present invention, the first parameter is used to quantify the difference between the amplitude information and the first threshold. The sending of the target signal to the external cooperating device includes: The terminal's interactive state machine is overwritten by the first instruction to bypass the observation window mechanism, and the target signal is sent to the collaborative care station, which is the external collaborative device, via a wireless link.

[0013] As a preferred embodiment of the present invention, the first feature data includes spatial acceleration vector and angular velocity mutation rate; the second feature data includes time-domain heart rate waveform and microvascular volume envelope sequence; and the third feature data includes physiological mutation rate features determined based on a sliding time window.

[0014] This invention also provides an interactive state tracking system based on human-computer collaboration, comprising: The multi-source data sensing module is used to acquire the first feature data and the second feature data; The gating module is used to extract the amplitude information of the first feature data. If the amplitude information is lower than the first threshold, a first identifier and a first parameter are generated, and feature extraction of the second feature data is triggered based on the first identifier. The arbitration engine module is used to obtain the second identifier of the terminal to determine the working range, and to determine the target scoring information based on the differentiated calculation strategy corresponding to the working range, combined with the first parameter and the extracted feature data. The backtracking execution module is used to send a target signal to an external collaborative device when the target score information meets the second threshold, and asynchronously start the trend verification and status rollback process of the sent signal.

[0015] The beneficial effects of this invention are: After acquiring first feature data and second feature data, this invention extracts the amplitude information of the first feature data and generates a first identifier and a first parameter when the amplitude information is lower than a first threshold. When the first identifier is captured, it triggers the extraction of third feature data from the second feature data. Then, based on the terminal's second identifier, the first parameter, and the third feature data, it determines the target scoring information. This effectively transforms weak, normally intercepted motion features into another feature channel for wake-up gating, solving the problem in the prior art where a fixed anti-accidental touch threshold causes weak, genuine emergency intentions from patients with chronic diseases to be intercepted as noise, leading to system failure. It also prevents emergency interaction commands from being ignored in extreme care scenarios on the cloud-based TCM platform. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0017] Figure 1 This is a flowchart illustrating the workflow of the intelligent interactive state tracking method based on human-machine collaboration of the present invention. Figure 2 This is a schematic diagram of the intelligent interactive state tracking system based on human-machine collaboration according to the present invention. Detailed Implementation

[0018] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0020] like Figure 1As shown, this application provides an intelligent interaction state tracking method based on human-machine collaboration. This method mainly operates in a network environment that includes edge monitoring terminals and external collaborative devices. Through cross-modal state correlation analysis and dynamic resource constraints, it solves the failure problem in complex and long-cycle physical collaboration scenarios where the true intent is erroneously intercepted due to the restricted actions of the operating subject.

[0021] At the monitoring nodes in the human-machine collaboration scenario, the system acquires continuous time-series data in real time. Specifically, the terminal's perception module continuously collects and outputs state information representing different dimensions of the operating subject, namely the first feature data and the second feature data, providing a basic data source for subsequent multimodal cross-validation.

[0022] The system inputs the received first feature data into the anti-accidental touch routing gating module for initial judgment. The system parses the first feature data within the preset time window, extracts its corresponding amplitude information, and compares the amplitude information with the first threshold set by the system. The first threshold represents the conventional interaction trigger baseline pre-configured by the system.

[0023] When the amplitude information is lower than the first threshold, the system breaks through the conventional static filtering logic that directly judges small fluctuations as high-frequency noise, actively intercepts the conventional intent inference of the channel, and generates a first identifier and a corresponding first parameter based on the interception action.

[0024] Specifically, regarding the setting of the first threshold, the system uses a composite numerical range based on the standard units of physical gravitational acceleration (g) and angular velocity. Conventional effective physical interaction actions (such as deliberately waving for help or tapping on a touch screen) typically generate spatial acceleration peaks between 0.8g and 1.5g.

[0025] Therefore, the specific value of the first threshold is configured as an acceleration scalar value of 0.5g-0.6g, or as an angular velocity mutation value of 30° / s.

[0026] When the operating subject in low-power mode suddenly becomes disabled, the physical acceleration generated by its subconscious weak twitching or struggle is mostly distributed in the extremely weak range of 0.2g-0.4g. By setting the first threshold baseline at around 0.5g, the system can accurately separate these real restricted actions from the conventional anti-accidental touch interception logic and use them as the gating conditions for triggering the first identifier and the first parameter.

[0027] In this logical chain, the first identifier serves as a state label indicating that the characteristics of the normal action are limited, while the first parameter is used to quantify the feature difference between the current amplitude information and the first threshold.

[0028] To unify the dimensions of subsequent calculations of multimodal data, the first parameter is calculated using a relative difference, specifically as follows: The difference between the first threshold (e.g., 0.5g) and the amplitude value of the extracted first feature data is calculated, and then the difference is divided by the first threshold. The value range of the first parameter obtained is limited to between 0 and 1.

[0029] Through this normalization method, when the operating subject is absolutely still, i.e., the amplitude value is 0, the first parameter is the maximum value of 1; when the action reaches the conventional anti-accidental touch interception benchmark, i.e., the amplitude value is equal to the first threshold, the first parameter is the minimum value of 0.

[0030] The first identifier is configured in the control flow as a pre-gating instruction to trigger multimodal joint evaluation. Only when the system control link captures the first identifier is it allowed to wake up and activate the deep analysis channel for the second feature data, and then calculate and extract the third feature data from the second feature data. By introducing a conditional triggering mechanism with the first identifier as a hard switch, the extraction of the second feature data can be avoided at all times under normal conditions, thus maintaining the low power consumption sleep baseline of the monitoring terminal while ensuring the accuracy of intent capture.

[0031] After extracting the third feature data, the system synchronously reads the second identifier, which reflects the current underlying operating load or energy efficiency status of the terminal. The system uses the second identifier as a prerequisite for branch routing, and combines it with the first parameter representing the degree of feature deviation and the newly extracted third feature data to perform multi-dimensional cross-fusion and logical inference, and outputs target score information to represent the current interaction intent intensity.

[0032] The system compares the calculated target score with a set second threshold for verification. This second threshold represents the emergency safety baseline under multimodal fusion. When the target score reaches or exceeds this second threshold, it indicates that although the current operating entity's physical actions are limited, it actually possesses a high degree of confidence in its collaborative intent after multimodal cross-verification. At this time, the system generates a target signal and transmits it to external collaborative devices to complete a rapid response to status takeover or emergency collaborative commands.

[0033] The second threshold is represented in the system kernel logic as a normalized dimensionless safety confidence baseline. The target score information is a comprehensive score output after multimodal cross-weighting or dimensionality reduction comparison, and its numerical domain is mapped to a normalized interval of 0-1 (or 0-100%).

[0034] The specific value range of the second threshold is usually configured between 0.75 and 0.85. In human-machine collaborative medical care scenarios, if the safety baseline is set too low (e.g., below 0.6), routine deep breathing or physiological fluctuations during sleep combined with extremely weak limb displacement can trigger takeover, leading to serious abuse of medical intervention resources. If it is set too high (e.g., above 0.95), under extremely weak conditions, even if the exponential amplification mechanism of physiological intention is activated, it is extremely difficult to break through this defense line.

[0035] Anchoring the second threshold between 0.75 and 0.85 achieves a better engineering balance between false alarm rate and false negative rate, ensuring the highly reliable issuance of emergency coordination commands.

[0036] Meanwhile, in order to ensure the compliant allocation of physical intervention resources, the system independently wakes up the asynchronous state backtracking verification process on the same time line as the target signal is issued. This backtracking verification process is independent of the main control business flow and is used to perform secondary verification of false positives (false alarms) on the events triggered by the issued signals within a preset time window, so as to ensure the safe closed loop of business flow and interactive intervention.

[0037] Furthermore, to ensure the continuity of the tracking link, the system reads and parses the second identifier of the terminal. The second identifier is a data tag representing the current hardware load or energy efficiency operating status of the terminal, such as the remaining power percentage parameter output by the battery management system (BMS) interface inside the terminal, or the instantaneous operating load rate of the processor scheduled by the system kernel. The system compares the specific value of the second identifier obtained with a preset resource baseline to determine the current working range of the terminal.

[0038] If the working interval is determined to be the first interval, it indicates that the terminal is currently in a state of sufficient hardware resources and computing power.

[0039] Specifically, after the system obtains the second identifier through the underlying interface, it performs a strict threshold comparison through the kernel's resource status monitor. When the second identifier indicates that the current battery percentage of the terminal is greater than or equal to the preset battery life warning line (e.g., the battery percentage is greater than or equal to 20%), and the hardware load information is within the system's normal safe computing power range (e.g., the average operating load rate of the central processing unit in the past second is less than or equal to 85%, and the chip core temperature has not triggered the system's underlying frequency reduction protection baseline), the system determines that the terminal is currently in the first interval.

[0040] At this point, the system control link activates the full cross-validation branch, and the system call processor extracts the first parameter output from the preceding process. Since this parameter accurately quantifies the mathematical difference between the magnitude of the intercepted physical action and the effective threshold, the system determines the weight information in reverse based on this difference feature.

[0041] Its core principle lies in dynamically converting the interception failure result of the action mode into an amplification factor of the physiological mode: The more severely physical interaction actions are restricted (i.e., the greater the degree to which the first parameter deviates from the normal range), the higher the amplification factor represented by the weight information assigned by the system. Subsequently, the system uses the determined weight information to perform weighted fusion processing on the second feature data in the entire time domain, and calculates the target score information through cross-modal data compensation. This mechanism maximizes the accuracy of capturing restricted true intentions in an environment where computing power allows.

[0042] If the working interval is determined to be the second interval, it indicates that the terminal is facing a physical extreme value of limited resources (e.g., the power is lower than the warning baseline or the underlying overload protection is triggered). If the high-frequency feature analysis in the entire time domain continues, it is very easy to exhaust the system resources and cause the monitoring terminal to fail silently. Therefore, the system suspends the high-consumption feature processing channel and triggers a lightweight dimension reduction and rollback mechanism.

[0043] The system extracts only a small, highly discriminative subset from the second feature data as the fourth feature data (e.g., retaining only the specific high-low frequency ratio sequence extracted by fast Fourier transform, while bypassing the complete time-domain waveform profile).

[0044] Then, the system performs a lightweight comparison operation between the fourth feature data and the pre-fixed physiological feature template to directly obtain the target score information.

[0045] To ensure the system has a reliable basis for execution during dimensionality reduction comparison, the data source, data structure, and comparison call logic of the physiological feature template are described in detail below: The physiological feature template is a static reference table that is solidified into the non-volatile storage area of ​​the terminal after statistical cluster analysis based on the real physiological fluctuation characteristics of the operator under critical conditions such as acute illness and disability caused by the historical empirical accumulation of the operator. The physiological feature template is represented as a key-value mapping data dictionary, which contains multiple-dimensional interval fields. The input key represents the numerical range in which the fourth feature data may fall, and the output value corresponds to the pre-defined benchmark score constant under different intervals. For example, when the extracted fourth feature data is the energy ratio of low frequency to high frequency, the specific mapping rule of the physiological feature template is as follows: If the energy ratio is less than 0.5, the corresponding benchmark score is 0.2; If the energy ratio is greater than or equal to 0.5 and less than 1.0, the corresponding benchmark score is 0.4; If the energy ratio is greater than or equal to 1.0 and less than 2.0, the corresponding benchmark score is 0.6; If the energy ratio is greater than or equal to 2.0 and less than 3.0, the corresponding benchmark score is 0.8; If the energy ratio is greater than or equal to 3.0, the corresponding benchmark score is 0.95.

[0046] Furthermore, the calibration method for the physiological characteristic template is as follows: cluster analysis is performed based on the true heart rate variability (HRV) data of 1,000 patients with acute onset, and the scoring constant is dynamically adjusted according to the coefficient of plus or minus 0.1 for different age groups of the operator.

[0047] During the comparison, the processor uses the fourth feature data extracted in real time as a query index to perform precise addressing and interval matching in the physiological feature template.

[0048] If the data falls precisely into a specific interval, the baseline scoring constant in the corresponding key-value pair is directly extracted as the target scoring information; if the data is at the critical position of an adjacent interval, the system outputs the smoothed final scoring information through basic linear interpolation calculation.

[0049] This fast lookup mechanism using a pre-built dictionary replaces complex continuous calculus or high-dimensional matrix operations, enabling medical-grade edge terminals to respond quickly in a low-power state safely and smoothly.

[0050] Furthermore, in the first interval where terminal computing power resources are abundant, the system does not adopt a simple linear substitution logic, but instead introduces a preset nonlinear exponential amplification function to dynamically adjust the weight of physiological characteristics in order to adapt to the intent verification requirements under extreme physical constraints.

[0051] Before the system performs weight mapping, the control link first obtains the extracted third feature data (such as the mutation rate feature value representing physiological fluctuations obtained in the previous steps) and compares it with the pre-configured third threshold, which represents the baseline of physiological characteristic fluctuations caused by basal metabolism or environmental interference in the normal state of the operating subject.

[0052] Only when the third feature data satisfies (i.e. reaches or exceeds) the third threshold, indicating that the currently captured physiological response belongs to an abnormal fluctuation with statistical significance, will the system activate the subsequent nonlinear weight calculation logic. By setting this precondition, weak physiological white noise can be effectively filtered out, preventing the system from mistakenly amplifying non-interactive, routine physiological changes.

[0053] When the third feature data satisfies the third threshold, the system processor calls a preset nonlinear exponential amplification function from the non-volatile storage area, inputs the first parameter into the function, calculates the physiological weight amplification coefficient, and outputs this coefficient as the weight information. A nonlinear exponential amplification function is used instead of a traditional linear scaling factor.

[0054] In extreme business scenarios such as sudden disability or falls, the range of physical movements of the operator often decreases sharply. In this case, linear compensation cannot push the fusion score above the safety baseline within a very short crisis window. However, exponential functions can ensure that the decision weight assigned to physiological data by the system increases non-linearly as physical movements become more restricted, thereby achieving unauthorized capture of emergency intentions.

[0055] Specifically, the preset nonlinear exponential amplification function is expressed by the following mathematical expression: ; Wherein, W1 represents the calculated physiological weight amplification coefficient, i.e., the weight information; P1 represents the first input parameter, used to quantify the difference between the amplitude information of the first feature data and the trigger threshold; K1 represents the basic scaling factor; K2 represents the sensitivity adjustment index; K0 represents the minimum anti-zero bias constant.

[0056] The value of K0 is usually configured to be between 0.1 and 0.3 to prevent the overall weight from dropping or the output from returning to zero when the first parameter P1 approaches zero, thus ensuring the robustness of the system in subsequent multiplication weighted calculations. The value of K1 is usually configured to be between 1.0 and 2.5, which is used to scale the output of the exponential term to a numerical space of the same order of magnitude as the first feature data, so as to prevent data overflow during cross-modal fusion. The value of K2 is usually configured to be between 0.5 and 1.5. Its specific value is derived from regression analysis of historical medical care empirical data and is used to control the slope of the exponential curve.

[0057] For different practical application scenarios, configuration instructions can be sent to the processor to dynamically adjust the value of the sensitivity adjustment index K2: In ordinary home care scenarios, the recommended value of K2 is 1.0; in the care scenario for disabled elderly, the recommended value of K2 is 1.2; and if it is for intensive care scenarios, K2 is increased to 1.5, so that the system has extremely high amplification sensitivity to slight differences.

[0058] In the specific working steps, the processor substitutes the real-time acquired P1 into the above formula to perform floating-point operations. Since the function adopts an exponential form with the natural constant e as the base, the value of the first parameter is positively correlated with the physiological weight amplification coefficient.

[0059] This means that the larger the value of the first parameter (i.e., the more severe the loss of amplitude when the physical action is intercepted and the weaker the action), the physiological weight amplification coefficient output by the system through this formula will increase exponentially.

[0060] Subsequently, the system directly applies the calculated W1 to the weighted fusion processing flow when the working interval is the first interval.

[0061] Specifically, the processor uses W1 as a dynamic adjustment factor to perform multiplicative weighted calculations on the extracted second feature data, thereby obtaining the compensated target score information.

[0062] The specific weighted fusion method is as follows: the fluctuation range of the time-domain heart rate waveform within a preset time window and the amplitude change of the microvascular volume envelope sequence at adjacent times are respectively processed by dimensionless processing, and then summed with a preset modal proportion coefficient (such as 0.7 for the heart rate end and 0.3 for the microvascular end). Finally, the summation result is multiplied by the physiological weight amplification coefficient to obtain the target score information.

[0063] Furthermore, regarding the third threshold corresponding to the third feature data, this threshold is used to filter out conventional physiological background white noise. When calculating the physiological mutation rate based on the time-domain heart rate waveform or microvascular volume envelope, the natural fluctuation of heart rate variability (basal metabolic fluctuation) in healthy individuals at rest is usually within the range of 5%-10%.

[0064] Therefore, the system configures the third threshold as a range of 15%-20% deviation from the steady-state baseline, wherein the steady-state baseline is defined as the average physiological characteristics of the user in a resting state for 30 consecutive minutes after wearing the terminal. Only when the physiological mutation rate captured by the system crosses the 15% non-steady-state threshold is it confirmed that the current physiological characteristics are caused by acute pathological stress, and only then is the aforementioned nonlinear exponential amplification function allowed to be activated.

[0065] Furthermore, the system acquires and parses the second identifier to quantify the current underlying physical load of the terminal.

[0066] Specifically, the second range is determined by the power percentage or hardware load information indicated by the second identifier.

[0067] In actual engineering configurations, the system kernel is configured with resource mapping rules: When the system detects that the battery percentage has dropped to a preset battery life warning line (e.g., the battery is below 20%), or the hardware load information has reached the system's high consumption threshold (e.g., the average operating load rate of the central processing unit in the past second is greater than 85%, or the chip core temperature exceeds the underlying frequency reduction protection baseline), the system determines that the terminal is currently in the second interval.

[0068] The second interval indicates that the terminal's hardware resources are in a critical state of high constraint, and the high-energy-consuming feature parsing channel must be suspended immediately.

[0069] In order to prevent the complex time-domain waveform deep analysis logic from exhausting the remaining resources and causing system failure, the control link bypasses the point-by-point comparison process of the full-time waveform.

[0070] The processor calls the built-in digital signal processing module to perform Fast Fourier Transform (FFT) processing on the continuously acquired second feature data (such as the original physiological sequence containing the time-domain heart rate waveform).

[0071] Through the above transformation process, the system can losslessly map redundant physiological signals that were originally distributed in the time dimension to the frequency domain space.

[0072] After completing the frequency domain mapping, the system uses pre-set bandpass filtering rules to selectively strip and extract subset features within a preset frequency domain range.

[0073] To ensure the accuracy of clinical physiological representations, the preset frequency range is strictly defined based on the medical statistical frequency band of heart rate variability (HRV), specifically including the low frequency (LF) band with a frequency between 0.04Hz and 0.15Hz, and the high frequency (HF) band with a frequency between 0.15Hz and 0.4Hz.

[0074] Furthermore, within the aforementioned defined preset frequency domain range, the system employs specific subset extraction rules to obtain the fourth feature data.

[0075] The specific extraction rules are as follows: The processor scans the power spectral density distribution within the low-frequency and high-frequency bands and selectively extracts the main frequency band signal whose energy accounts for more than 60% of the total energy of the entire band (i.e., the HRV standard analysis band of 0.04Hz-0.4Hz). Alternatively, the system can directly lock the center frequency of the power spectral density peak within the frequency band and extract the spectral data within a window range of ±0.02Hz before and after the center frequency.

[0076] The system uses the high-discrimination frequency domain information (such as the energy ratio of low frequency to high frequency) extracted by the above extraction rules as the fourth feature data.

[0077] By reducing the judgment criteria from complex time-domain waveform features to extremely simple frequency-domain subset features, the system shuts down high-computational-power-consuming channels at the underlying logic level. Without performing high-order matrix operations or complex iterations, it ensures that even under extreme physical boundaries such as power depletion or computing power overload, the terminal can still maintain the basic recognition ability of critical physiological intentions with very little feature data. This minimizes the processor's operating load and power consumption, and safely and stably guarantees the real-time response capability and business continuity of medical-grade edge terminals.

[0078] Furthermore, in the actual operating environment of human-machine collaboration, each independent sensor module is at risk of single-modal data distortion due to physical detachment, local electromagnetic interference, or user struggle.

[0079] If the data from different modalities diverge significantly at this point (for example, an inertial sensor determines that the system is absolutely stationary, while a bioelectric sensor shows extremely abnormal high-frequency fluctuations), directly weighting and fusing these data can easily trigger false alarms in the system.

[0080] To mitigate the aforementioned risks, before executing the fusion logic, the system extracts and calculates the modality pointing information corresponding to the first feature data and the third feature data, respectively. The modality pointing information represents the single-modal intent confidence value output by each independent data extraction channel after normalization processing. The control link synchronously acquires the modality pointing information of different modalities and calculates the absolute difference between them, which serves as a deviation reflecting the multimodal collaborative confidence.

[0081] Specifically, the modal orientation information of the first feature data is calculated as follows: the peak acceleration within a preset time window is divided by the acceleration reference value of a normal effective interaction (such as 0.8g) to obtain a first ratio, and the smaller of the first ratio and the value 1 is taken as the modal orientation information of the first feature data. The modality orientation information of the third feature data is calculated as follows: the physiological mutation rate feature is divided by the physiological mutation baseline value at acute onset (e.g., 0.3) to obtain the second ratio, and the smaller of the second ratio and the value 1 is taken as the modality orientation information of the third feature data.

[0082] The control link synchronously acquires the modality pointing information of different modes and calculates the absolute value of the difference between the two modality pointing information, which is used as the deviation reflecting the confidence of multimodal collaboration.

[0083] Subsequently, the system compares the calculated deviation with a built-in preset conflict threshold. If the deviation exceeds the preset conflict threshold, it indicates that there is a significant feature mutual exclusion phenomenon between the current action mode and the physiological mode. At this time, the system triggers the system error prevention and correction logic, introduces a conflict penalty factor to reduce the weight of the originally planned output target score information, and prevents noise data from single-point sensors from polluting the final decision output by pulling down the comprehensive score with high conflict risk.

[0084] In practical engineering implementation, the processor calls the following preset penalty correction function to perform the penalty correction operation: ; ; Wherein, Q1 represents the target score information before correction (i.e., the preliminary score originally planned to be obtained through the aforementioned fusion or comparison steps); Q2 represents the target score information after correction, which is used as the final basis for subsequent comparison decisions; F1 represents the calculated conflict penalty factor; V1 represents the deviation of the modality pointing information of different modalities; T1 represents the preset conflict threshold; K3 represents the penalty intensity coefficient; and ε1 represents the minimum anti-zero bias constant.

[0085] The value of T1 is usually set between 0.3 and 0.4. Under real sudden extreme working conditions, the cross-modal data fluctuation caused by action contraction and rapid physiological changes is usually within 30%. Once the deviation exceeds this range, there is an extremely high probability of single-channel physical failure or external interference. The value of K3 is configured to be between 0.2 and 0.5, which is used to control the linear slope of the target score when the penalty factor is pulled down. The reason for setting this coefficient is to ensure the smoothness of the system correction process and avoid the sudden change in score when the deviation just exceeds the threshold, thereby preventing the system state machine from jumping and collapsing. The preferred value of ε1 is 0.0001. Since the formula involves division, this constant is added as a bottom-level catch-all parameter in the denominator to prevent fatal system-level errors such as division by zero when performing small numerical calculations or hardware floating-point anomalies.

[0086] In summary, when the processor determines that the deviation V1 is greater than the threshold T1, it substitutes each variable into the above formula to calculate the conflict penalty factor F1 which is less than 1, and then performs a floating-point multiplication operation on it with the initially determined Q1.

[0087] Through this modified formula, the system converts the excess portion of the deviation overflow into penalty weights, reducing the artificially high scores caused by unidirectional modal distortion, making it difficult for them to exceed the second threshold that triggers unauthorized takeover. This mechanism enhances the system's fault tolerance and robustness in tracking decisions under complex physical environments.

[0088] Furthermore, while the system sends out the target signal, the kernel scheduler wakes up an independent security sandbox thread at the underlying level. Without blocking the front-end data acquisition and real-time early warning link, the thread aligns and splices the continuously collected first feature data and second feature data within a preset time window (e.g., 30-60 seconds after the instruction is issued), and caches it as joint sequence data in the terminal's non-volatile storage area or an independent memory queue.

[0089] Specifically, the joint sequence data is a spliced ​​sequence of the first feature data (such as acceleration amplitude) and the second feature data (such as heart rate value) that are strictly aligned according to a set time beat (such as 100 milliseconds).

[0090] After caching is completed, the system calls the processor to perform trend fitting processing on the joint sequence data to determine whether the characteristic mutations that trigger the warning have pathological continuity characteristics.

[0091] To avoid overloading the system's computing power due to complex high-order polynomial fitting, this application employs a lightweight linear trend decay verification algorithm. The processor executes the following preset trend verification formula through built-in operator logic: ; Wherein, R1 represents the output result after trend fitting, i.e., the state decay gradient value; P1 represents the characteristic peak value of the joint sequence data at the moment the target signal is triggered (i.e., the extreme value of the sudden change that triggers the warning), specifically defined as the peak value of the acceleration amplitude within 5 seconds before and after the trigger moment; A1 represents the characteristic mean value of the joint sequence data after moving average filtering within the entire preset time window, wherein the moving average filtering adopts a rectangular time window of 3 seconds and the sliding step size is set to 100 milliseconds; T2 represents the time span length of the preset time window; K4 represents the trend normalization coefficient; ε2 represents the minimum anti-zero bias constant.

[0092] The value of K4 is usually configured to be between 1.0 and 5.0, which scales the decay gradient under different modal dimensions to a standard logical decision range, making it easier for the system to perform fast floating-point threshold comparison. ε2 is set to 0.001. Since the calculation involves time span as the denominator of the division, this constant is introduced as a fallback parameter to avoid division by zero overflow error caused by abnormal timestamp recording or jump in the underlying clock cycle.

[0093] If the target signal is triggered by a real fall event, the combined physiological and motor state of the operator will remain in the abnormal range. At this time, the mean A1 within the time window will be close to or maintained near the peak P1, and the calculated R1 will be close to zero or positive. Conversely, if the trigger is a false positive event caused by transient white noise or accidental local compression, the feature data will quickly fall back to the normal baseline after the trigger, resulting in the mean A1 being much smaller than the peak P1. In this case, the calculated R1 will show a significant negative value.

[0094] The control link compares the calculated processing result R1 with a preset attenuation threshold (e.g., -2.5).

[0095] If R1 is less than the attenuation threshold, the system determines the target signal as a false trigger signal based on the processing result. Once it is confirmed that an overreaching misjudgment has occurred, the system does not block the physical warnings issued earlier (in order to ensure the absolute speed of crisis response), but immediately generates a second instruction.

[0096] The second instruction carries a status downgrade or revocation flag. The system sends the second instruction to the external collaborative device through the wireless gateway, notifying the remote nursing station to mark the event as system white noise, thereby safely and compliantly deactivating the collaborative state corresponding to the target signal.

[0097] Furthermore, when the system performs continuous data acquisition in the underlying data sensing module, it simultaneously activates heartbeat monitoring for the integrity of the data stream. If the second feature data is continuously empty within a preset period (e.g., multiple consecutive hardware sampling clock cycles, or within a set fixed time observation window) (specifically manifested as continuous loss of data packets on the data bus, continuous return of the channel to zero level, or signal-to-noise ratio lower than the hardware sleep baseline), the system determines that an actual fault has occurred in the current physiological feature acquisition link.

[0098] To prevent the multimodal arbitration engine from falling into a thread dead or service failure state due to endless waiting for cross-modal data alignment, the control link suspends the regular cross-evaluation branch and immediately generates a third identifier, which is configured in the system kernel as the highest priority interrupt label representing a local sensor failure.

[0099] After generating the third identifier, the system does not directly block the business flow or simply report an error. Instead, it performs a composite verification by combining the historical state accumulated in the previous anti-accidental touch process. The processor calls the cache queue in the storage area to extract and count the distribution frequency of the first parameter in the recent sliding window.

[0100] As mentioned earlier, the first parameter accurately quantifies the magnitude difference when a physical interaction action is judged to be intercepted, and its timing and frequency can objectively reflect the density of the weak actions performed by the operating subject.

[0101] In the specific control logic, the system determines whether the discrete distribution frequency of the first parameter exceeds the set frequency baseline. If the distribution frequency shows abnormal high-frequency aggregation characteristics, it means that the operator is in a state of struggle with frequent exertion but limited range of motion at the same time as the physiological sensor falls off (such as subconscious convulsions during a sudden illness). At this time, the risk of the real emergency intention being missed by the system is extremely high, and the system will immediately trigger the audible and visual alarm of the terminal (such as activating an 80dB buzzer and controlling the LED indicator to flash red).

[0102] In the disconnection monitoring logic of the underlying hardware, the preset period is defined as the continuous counting of the absolute clock span or the hardware sampling beat as the determination condition for the second feature data to be continuously empty.

[0103] Generally, considering that wearable sensors may experience transient level fluctuations or poor contact for hundreds of milliseconds during vigorous exercise, the specific value of the preset period should not be set too short.

[0104] The system configures the preset period in the kernel driver as a continuous time window of 3-5 seconds, or equivalent to 30-50 consecutive underlying hardware sampling clock cycles.

[0105] If the data bus fails to capture any valid physiological simulation signal within the preset period, the system confirms from the hardware level that an irreversible actual detachment failure has occurred, which serves as a rigorous basis for triggering the third identifier and activating the local audible and visual fallback alarm.

[0106] Furthermore, the first parameter is used to quantify the difference between the amplitude information and the first threshold. In the actual feature parsing chain, this parameter accurately represents the degree of loss of the physical action amplitude caused by the sudden limitation of physiological function when the operating subject initiates a real emergency interaction from the normal effective baseline. By quantifying and extracting this difference feature, the system can reverse the negative result that was originally judged as invalid interception into the logical cornerstone of triggering and amplifying physiological feature verification, fundamentally breaking the linear judgment limitation of relying solely on physical amplitude.

[0107] Once the system completes multimodal cross-fusion and determines that the target scoring information meets the set second threshold, the control link immediately enters the highest priority emergency response execution phase.

[0108] At this time, in order to prevent the high-confidence warning signal from being intercepted a second time by the local anti-accidental touch logic, the system calls the underlying scheduler to generate the first instruction. The processor uses the first instruction to directly overwrite the working rules of the interactive state machine currently running on the terminal in memory. Through this overwrite operation, the system bypasses the observation window mechanism, that is, skips the long time sequence verification and data buffering waiting process that the state machine is required by default when facing abnormal short signals.

[0109] Under the default operating rules, the interactive state machine adopts a 5-second observation window mechanism, meaning that an early warning is only triggered when an abnormal interaction signal lasts for more than 5 seconds. The first instruction specifically writes a preset forced trigger value (such as 0x01) to the terminal's control register (such as register 0x1A). This operation forces the observation window duration of the underlying state machine to 0 seconds, thereby directly skipping the timing verification process and achieving signal pass-through.

[0110] After successfully bypassing local defense interception, the system immediately wakes up and takes over control of the communication baseband, opens a data pass-through channel with no delay, and controls the internal radio frequency transmission module to directly transmit the target signal indicating status takeover or emergency warning to the collaborative care station, which is the external collaborative device, through a pre-paired wireless link (such as Bluetooth Low Energy link, Wi-Fi or cellular network).

[0111] Furthermore, the system's data sensing module performs synchronous sampling in two dimensions: physical space and physiological characteristics. Specifically, the first feature data includes spatial acceleration vector and angular velocity mutation rate. In actual hardware configuration, these data are output by a multi-axis inertial measurement unit (IMU) worn at the end of the operating subject (such as the wrist or finger).

[0112] Among them, the spatial acceleration vector is used to represent the absolute force amplitude and displacement direction of physical interaction actions in the three-dimensional coordinate system; the angular velocity mutation rate is used to quantify the transient impact energy when the limb twists or flips. By combining these two underlying data, the system can accurately depict the physical dynamic profile of the operator during normal interaction or sudden fall.

[0113] On the physiological sensing channel side, the second feature data includes time-domain heart rate waveforms and microvascular volume envelope sequences. These data are typically acquired continuously by optical or bioelectric sensing units (such as PPG photoplethysmography sensors). The time-domain heart rate waveform directly reflects the instantaneous stress state of the operator's autonomic nervous system; the microvascular volume envelope sequence represents the slow trend of local blood perfusion caused by peripheral vasoconstriction or vasodilation. When the operator encounters a crisis situation such as an acute illness attack, it is often accompanied by abnormal fluctuations in heart rate and a sudden drop in peripheral microvascular perfusion. These two types of sequence data provide high-confidence internal physiological evidence for the system when physical movement is restricted.

[0114] In order to transform the continuous underlying physiological sequence into a logical judgment basis that can be directly called by the routing gating, the third feature data is configured to include physiological mutation rate features determined based on a sliding time window. Instead of performing indiscriminate calculations on the global physiological data, the system opens a sliding time window of a certain length in memory through the processor and performs discretization sampling and variance concatenation operations on the second feature data that falls within the window.

[0115] To ensure the robustness of the computational logic and its direct execution by a computer in engineering applications, the system calls a built-in operator to execute the following preset joint mutation rate quantification formula: ; Wherein, M1 represents the calculated physiological mutation rate characteristic (i.e., the third characteristic data); N1 represents the total number of discrete sampling points within the sliding time window; X i X represents the instantaneous value of the i-th time-domain heart rate waveform data point within the sliding time window; a Y1 represents the arithmetic mean of all heart rate waveform data points within the sliding time window; Y0 represents the amplitude of the microvascular volume envelope sequence acquired at the current moment; Y1 represents the amplitude of the microvascular volume envelope sequence acquired at the previous moment; T3 represents the set duration of the sliding time window; K5 represents the first mode convergence coefficient; K6 represents the second mode convergence coefficient; ε3 and ε4 both represent the minimum anti-zero bias constant.

[0116] The value of K5 is usually configured between 0.6 and 0.8, while the value of K6 is configured between 0.2 and 0.4. During the very short window period of sudden disability, the response speed and signal-to-noise ratio of heart rate variability (represented by the standard deviation term in the first half of the formula) are significantly higher than the slow change in peripheral microvascular volume (represented by the term in the second half of the formula). Therefore, a higher convergence coefficient is given to the heart rate term to improve the sensitivity of capturing sudden situations. The fixed values ​​of ε3 and ε4 are configured as 0.0001. These are introduced as hardware-level anti-crash parameters to strictly avoid division-by-zero overflow errors caused by abnormal T3 readings due to system clock cycle jumps or by N1 not being accumulated during memory sampling queue initialization.

[0117] Specifically, when the first interactive action feature is intercepted by the anti-accidental touch module and generates the first identifier, the control link immediately wakes up the above-mentioned calculation channel. The processor performs floating-point operations on the heart rate fluctuation and the slope of the microvascular volume within the sliding time window, reducing the complexity of the continuous physiological sequence to a scalar value M1 representing the absolute physiological stress intensity.

[0118] Furthermore, such as Figure 2 As shown, the present invention also provides an interactive state tracking system based on human-machine collaboration. This system, as the hardware and logical entity that executes the aforementioned intelligent interactive state tracking method, is usually integrated into wearable edge monitoring terminals such as medical care devices. The overall architecture of the system is physically divided into a sensor array for front-end signal capture, a core processor for scheduling, and a communication baseband for remote interaction. In terms of logical control flow, it is manifested as multiple independent modules that cooperate with each other.

[0119] The system includes a multi-source data sensing module, which is directly connected to the underlying hardware sensors via an internal high-speed data bus to acquire first and second feature data. In actual hardware deployment, this module includes an inertial measurement unit (such as a three-axis or six-axis accelerometer and gyroscope integrated on the terminal motherboard) for outputting motion vector signals, and a photoelectric or bioelectric sensing unit (such as a PPG photoplethysmography sensor attached to the skin surface of the operating subject) for synchronously outputting physiological signs. The weak analog signals acquired by the above sensors are processed by a high-precision analog-to-digital converter (ADC) and stored directly as a continuous digital sequence in the terminal's direct memory access (DMA) cache, providing a reliable basic data source for subsequent multimodal cross-validation.

[0120] The system also includes a gating module, which is typically deployed in the digital signal processor (DSP) of the terminal or the underlying logic control unit of the main control chip. This module is used to perform hardware-level initial decision-making to prevent accidental touches. The module extracts the amplitude information of the first feature data from the buffer and calls the built-in comparator circuit to compare it with a first threshold stored in the secure storage area.

[0121] If the amplitude information is lower than the first threshold, the gating module determines that the current physical action is limited or belongs to a weak fluctuation, and then generates a first identifier and a first parameter. The first parameter (i.e., a value that precisely quantifies the difference between the amplitude information and the first threshold) is written into a designated register. At the same time, the system triggers an internal hardware interrupt signal based on the first identifier to wake up the physiological feature extraction channel in a low-power sleep state, thereby realizing deep feature extraction of the second feature data. This hardware interrupt-based pre-triggering mechanism effectively avoids the ineffective consumption of computing power for the full modality under normal action amplitude.

[0122] The system further includes an arbitration engine module, which is the core processing unit for realizing cross-modal state reverse correlation and dynamic resource constraints.

[0123] During actual operation, this module communicates with the battery management system (BMS) or kernel scheduler inside the terminal through the system-level interface (API) to read the second identifier used to indicate the terminal's current remaining power or instantaneous computing load.

[0124] Based on the obtained second identifier, the system determines the current working interval of the underlying hardware (i.e., the first interval with sufficient computing power or the second interval with limited resources). Subsequently, the arbitration engine module, based on the differentiated calculation strategy corresponding to the working interval, extracts the first parameter temporarily stored in the previous register as a weight amplification factor, and performs lightweight dimensionality reduction comparison processing by weighted fusion or bypass full-time domain extraction in combination with the newly extracted feature data, and outputs target score information that reflects the intensity of the current operator's urgent interaction intent.

[0125] The system is also equipped with a backtracking execution module, which is independent of the main control business flow and is managed by the operating system's asynchronous scheduling thread.

[0126] When the target score information output by the arbitration engine module reaches or exceeds the second threshold representing the emergency safety baseline, the backtracking execution module immediately takes over the system's communication radio frequency link and sends the target signal to external collaborative devices (such as remote multi-target collaborative nursing stations) to achieve seamless emergency takeover.

[0127] Meanwhile, the module allocates an independent security sandbox memory within the terminal's non-volatile storage area to asynchronously initiate trend verification and state rollback processes for the transmitted signals. By performing lightweight trend decay fitting on the feature sequences continuously written to the sandbox within a preset time window, if it is confirmed that the earlier trigger was a false positive event caused by transient noise, the module calls the wireless gateway to resend the coordination cancellation or state degradation command. Thus, while ensuring rapid response under extreme conditions, a compliant and secure closed loop for physical intervention resources is constructed at the underlying level.

[0128] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for tracking the intelligent interaction state based on human-machine collaboration, characterized in that, include: Obtain the first feature data and the second feature data; Extract the amplitude information of the first feature data, and generate a first identifier and a first parameter based on whether the amplitude information is lower than a first threshold; If the first identifier is captured, the third feature data of the second feature data is extracted; Based on the terminal's second identifier, the first parameter, and the third feature data, the target scoring information is determined. If the target score information meets the second threshold, a target signal is sent to an external collaborative device, and asynchronous state backtracking verification is triggered.

2. The intelligent interaction state tracking method based on human-machine collaboration according to claim 1, characterized in that, The determination of target scoring information includes: The current working range of the terminal is determined based on the second identifier; When the working interval is the first interval, weight information is determined based on the first parameter, and the second feature data is weighted and fused based on the weight information to obtain the target score information; When the working interval is the second interval, the fourth feature data is extracted from the second feature data, and the target score information is obtained based on the comparison result of the fourth feature data and the physiological feature template.

3. The intelligent interaction state tracking method based on human-machine collaboration according to claim 2, characterized in that, The step of determining the weight information based on the first parameter includes: When the third feature data meets the third threshold, the first parameter is input into a preset nonlinear exponential amplification function to calculate the physiological weight amplification coefficient, which is used as the weight information. The value of the first parameter is positively correlated with the physiological weight amplification coefficient.

4. The intelligent interaction state tracking method based on human-machine collaboration according to claim 2, characterized in that, The extraction of the fourth feature data from the second feature data includes: Perform a Fast Fourier Transform on the second feature data to extract a subset of features within a preset frequency domain range as the fourth feature data; The second range is determined by the power percentage or hardware load information indicated by the second identifier.

5. The intelligent interaction state tracking method based on human-machine collaboration according to claim 1, characterized in that, Before determining the target scoring information, the method further includes: Calculate the modality pointing information corresponding to the first feature data and the third feature data respectively; If the deviation of the modality pointing information in different modalities exceeds a preset conflict threshold, a conflict penalty factor is introduced to correct the target scoring information.

6. The intelligent interaction state tracking method based on human-machine collaboration according to claim 1, characterized in that, The triggering of asynchronous state backtracking verification includes: Cache the combined sequence data of the first feature data and the second feature data within a preset time window; The joint sequence data is subjected to trend fitting processing, and the target signal is determined as a false trigger signal based on the processing result. If the target signal is the false trigger signal, a second instruction is generated and sent to the external collaborative device to cancel the collaborative state corresponding to the target signal.

7. The intelligent interaction state tracking method based on human-machine collaboration according to claim 1, characterized in that, After acquiring the first feature data and the second feature data, the method further includes: If the second feature data is continuously empty within a preset period, a third identifier is generated; Based on the distribution frequency of the third identifier and the first parameter, the terminal is triggered to issue an audible and visual alarm.

8. The intelligent interaction state tracking method based on human-machine collaboration according to claim 1, characterized in that, The first parameter is used to quantify the difference between the amplitude information and the first threshold; The sending of the target signal to the external cooperating device includes: The terminal's interactive state machine is overwritten by the first instruction to bypass the observation window mechanism, and the target signal is sent to the collaborative care station, which is the external collaborative device, via a wireless link.

9. The intelligent interaction state tracking method based on human-machine collaboration according to claim 1, characterized in that, The first feature data includes spatial acceleration vector and angular velocity mutation rate; the second feature data includes time-domain heart rate waveform and microvascular volume envelope sequence; the third feature data includes physiological mutation rate features determined based on a sliding time window.

10. An interactive state tracking system based on human-machine collaboration, characterized in that, include: The multi-source data sensing module is used to acquire the first feature data and the second feature data; The gating module is used to extract the amplitude information of the first feature data. If the amplitude information is lower than a first threshold, a first identifier and a first parameter are generated, and feature extraction of the second feature data is triggered based on the first identifier. The arbitration engine module is used to obtain the second identifier of the terminal to determine the working range, and to determine the target scoring information based on the differentiated calculation strategy corresponding to the working range, combined with the first parameter and the extracted feature data. The backtracking execution module is used to send a target signal to an external collaborative device when the target score information meets the second threshold, and asynchronously start the trend verification and status rollback process of the sent signal.

Images