Action estimation method for task sufficient state under missing sensor data, terminal and storage medium

Abstract

The present application relates to the technical field of state estimation, and discloses a task sufficient state action estimation method under missing sensor data, a terminal and a storage medium, the method comprises the following steps: mapping multiple sensor observations to a latent space by using a task supervision encoder to generate a dynamic prior at the current time, then generating a prior injection coefficient, adopting a measurement preserving posterior update mechanism, limiting the disturbance of the prior to the measurement state when the observation is reliable, and using the dynamic prior to compensate when the observation is weak or missing seriously, so as to obtain an accurate posterior state. The present application can obtain a more stable state representation for downstream identification, classification or prediction tasks under the condition of sensor missing, weak observation, limited sensor cost or unstable observation factors.

Description

Technical Field

[0001] This invention relates to the field of state estimation technology, and in particular to a method, terminal, and computer-readable storage medium for estimating the action of a task in a state with insufficient sensor data. Background Technology

[0002] In recent years, with the development of artificial intelligence, the Internet of Things, wearable devices, mobile sensing systems, and industrial intelligent monitoring technologies, multi-sensor sequence data has been widely applied to tasks such as human activity recognition, health monitoring, vehicle status estimation, industrial equipment fault prediction, and event detection. These systems typically rely on multiple sensors to continuously collect observational data and use multi-sensor information fusion models to determine the system state or downstream task category.

[0003] While existing multi-sensor intelligent sensing methods can achieve good recognition or prediction results under ideal conditions where sensor data is complete, sampling is stable, and device location is fixed, sensor observations are often incomplete, unstable, and costly in real-world deployment scenarios. On the one hand, sensors may produce missing or abnormal observations due to reasons such as device disconnection, communication interruption, changes in wearing position, obstruction, asynchronous sampling, environmental noise interference, or channel contamination. On the other hand, sensor acquisition is also limited by factors such as energy consumption, latency, bandwidth, computing resources, and privacy, meaning the system cannot always collect all sensor data.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a method, terminal, and computer-readable storage medium for estimating the action of a task with sufficient state under conditions of missing sensor data. This aims to solve the problems of existing technologies, which make it difficult to obtain a sufficiently effective state representation for downstream tasks under conditions of missing observations, weak observations, or cost constraints, and the inaccurate priors that can easily damage reliable observations, thus leading to errors in the prediction results of the task.

[0006] To achieve the above objectives, the present invention provides a method for estimating the action of a task in a sufficiently large state with missing sensor data. The method includes the following steps: Multiple usable observation data collected by multiple human body sensors on the target object are acquired, each of the usable observation data is encoded into a latent space to obtain a corresponding latent representation, and all the latent representations are fused to obtain the current measurement state; The historical posterior states under the obtained task-sufficient state are used to predict the current dynamic trend of the target object based on the historical posterior states and the dynamic operators introduced into the potential space, thereby obtaining the current prior state. A constraint optimization function for the current posterior state is constructed using the current measurement state and the current prior state. Prior injection coefficients are constructed based on the constraint optimization function, and the current posterior state is generated based on the prior injection coefficients. Based on multiple detection indicators of each of the human body sensors, the current posterior state is subjected to task state anomaly detection. The current posterior state is repaired by combining the prior injection coefficient to obtain the target posterior state. The target posterior state is then subject to amplitude limitation. The target posterior state within a preset range is used to predict the activity type of the target object.

[0007] Optionally, the action estimation method for a task with sufficient state under missing sensor data, wherein acquiring multiple available observation data collected by multiple human sensors on the target object, encoding each available observation data into a latent space to obtain a corresponding latent representation, and fusing all the latent representations to obtain the current measurement state, specifically includes: Based on the acquisition command from the previous moment, control multiple human body sensors to collect multiple available observation data of the target object in the target task, and add a corresponding availability mask for each type of available observation data; Each available observation is input into the corresponding branch encoder for feature extraction, thereby encoding each available observation into the latent space to obtain the corresponding latent representation: ; in, Indicates the first The available observation data in the first The potential representation of time, Indicates the first One branch encoder, Indicates the first Personal body sensors in the first Available observational data at time, This represents the set of sensor group numbers that currently provide available observation data; Each latent representation is decomposed into a shared task state and a private residual state. Based on all the shared task states and all the private residual states, all the latent representations are fused using a measurement state fusion function to obtain the current measurement state. ; in, Indicates the first Time of the first A shared task state between individual human sensors and downstream tasks is used to participate in cross-sensor fusion and dynamic prediction. Indicates the first Time of the first Private residual states in personal body sensors; ; in, Indicates the current measurement status. Represents the measurement-state fusion function. Indicates availability mask.

[0008] Optionally, the action estimation method for a task-sufficient state with missing sensor data, wherein the historical posterior state obtained under the task-sufficient state is used to predict the current dynamic trend of the target object based on the historical posterior state and the dynamic operators introduced into the latent space to obtain the current prior state, specifically includes: Obtain the complete observation history at the current moment, and use the complete observation history to calculate the risk of the first task: ; in, Indicates the first The primary task and risk at all times This represents a function indicating the risk of a task. This represents the task loss function. This represents the output function of a reference mission based on a complete observation history. Indicates up to the number The complete observation history of the moment, Indicates the downstream task label; Obtain the predicted dynamic posterior state from the previous time step, and construct the second task risk based on the dynamic posterior state: ; in, Indicates the first The second task risk at any given moment This represents the task output function based on the posterior state. Indicates the dynamic posterior state; Based on the first task risk and the second task risk, determine whether the dynamic posterior state is a historical posterior state of a task-sufficient state: If the difference between the first task risk and the second task risk is not greater than a first non-negative threshold, or the conditional mutual information between the first task risk and the second task risk is not greater than a second non-negative threshold, then the dynamic posterior state is determined to be a historical posterior state of a task-sufficient state. ; ; in, This represents the first non-negative threshold. Conditional mutual information refers to mutual information under dynamic posterior state conditions. and The amount of information shared between them Indicates the second non-negative threshold; A dynamic operator is introduced into the latent space. Based on the historical posterior state and the dynamic operator, the current dynamic trend of the target object is predicted to obtain the current prior state at the current moment. ; in, Indicates the first The current prior state at any given moment. This represents a dynamic operator.

[0009] Optionally, the action estimation method for a task-sufficient state with missing sensor data, wherein the historical posterior state obtained under the task-sufficient state is used to predict the current dynamic trend of the target object based on the historical posterior state and the dynamic operators introduced into the latent space to obtain the current prior state, further includes: A normalized dynamic innovation magnitude is constructed based on the current measurement state and the current prior state to measure the consistency between the current measurement state and the current prior state: ; ; in, Indicates the first The dynamic innovation vector between the current measured state and the current prior state at time step [time]. Indicates the first The current measurement state at any given time. Indicates the first The current prior state at any given moment. Indicates the first The normalized dynamic innovation magnitude between the current measured state and the current prior state at time t. Indicates the first The potential state dimension at any given moment; The reliability of the current measurement state is constructed based on the normalized dynamic innovation magnitude, observation noise, historical consistency, task confidence, outlier observation score, and domain offset score. ; in, Indicates the first Reliability at any moment This represents the activation function. All of these represent learnable parameters. Indicates the first The current effective missing rate at any given time. Indicates the first Observation noise at any given moment Indicates the first Historical consistency of time Indicates the first Task confidence at any given time Indicates the first The score of abnormal observations at any given time. Indicates the first Domain offset or sensor misalignment fraction at any given time; ; ; in, Indicates the first The largest category predicted by the current measured state at time step. express In category The output logic on, express In category The output logic on, The set of categories representing the current measurement state.

[0010] Optionally, the action estimation method for a sufficient state of a task under missing sensor data, wherein the step of constructing a constrained optimization function for the current posterior state using the current measured state and the current prior state, constructing prior injection coefficients based on the constrained optimization function, and generating the current posterior state based on the prior injection coefficients specifically includes: Construct a constrained optimization function for the current posterior state using the current measurement state and the current prior state: ; in, Indicates the first The current posterior state at time t. Indicates the first The measurement state weights at time t. This represents the optimized posterior state. Indicates the first The current measurement state at any given time. Indicates the first Dynamic prior weights at time points, express and The square of the norm between express and the current prior state The square of the norm between; Based on the constraint optimization function, determine the constraint conditions: ; ; ; in, This represents the maximum limit parameter. express and The norm between, express and when The norm between, Indicates the first Reliability parameters at any given time. This indicates an extremely small positive number that prevents the denominator from being zero; A prior injection ratio is constructed based on the measured state weights and the dynamic prior weights, and prior injection coefficients are constructed based on the prior injection ratios: ; ; in, Indicates the first Prior injection ratio at time, Indicates the first Prior injection coefficients at time step, Represents the shearing function; Using the current measurement state as the anchor point, construct the current posterior state based on the prior injection coefficients: .

[0011] Optionally, the action estimation method for a sufficient task state under missing sensor data, wherein the step of performing task state anomaly detection on the current posterior state based on multiple detection indicators of each of the human body sensors, and repairing the current posterior state by combining the prior injection coefficients to obtain the target posterior state, specifically includes: Obtain the task reward, dynamic innovation reward, redundancy penalty, and multidimensional cost vector for each human sensor. For each human sensor, construct a selection score based on the task reward, dynamic innovation reward, redundancy penalty, and multidimensional cost vector. ; in, Indicates the first Personal body sensors in the first The choice of time to score points They represent The weight, Indicates the first Personal body sensors in the first Real-time task rewards Indicates the first Personal body sensors in the first Dynamic innovation benefits at all times Indicates the first Personal body sensors in the first Redundancy penalty in time Indicates the first Personal body sensors in the first The multidimensional cost vector at time step. Indicates transpose; ; ; in, This indicates uncertainty regarding the task. This represents the expected decrease in mission uncertainty. Indicates the first Personal body sensors in the first The expected dynamic innovation magnitude at any given moment; ; in, They represent the first The energy cost, latency cost, bandwidth cost, privacy cost, and computing cost of personal body sensors; Based on all the human body sensors, a sensor relationship graph is constructed to obtain the relationship strength between each of the human body sensors: ; in, Indicates the first Time of the first Personal body sensors and the The strength of the relationship between individual human sensors This represents the activation function. express The weighted value, Indicates the first Time of the first Personal body sensors and the Differences in measurement states between individual human body sensors express The weighted value, Indicates from the first Time to the The first moment Personal body sensors and the Differences in dynamic innovation vectors among individual human sensors; The acquisition process of the human body sensor is monitored based on the sensor relationship diagram, and a selection problem representation is constructed under multidimensional budget conditions based on all the selection scores, thus constructing constraints for the selection of the human body sensor: ; ; in, Indicates the first Time constraints, Indicates the first Personal body sensors in the first The problem of choosing a time is represented as follows: Indicates the number of human body sensors. Indicates the first Categories of personal body sensors Indicates multidimensional budgeting, Indicates the constraints for selecting human body sensors; An anomaly suppression gate is constructed for each time step, and the anomaly suppression gate is introduced into the corresponding prior injection coefficient to obtain the effective prior injection coefficient: ; in, Indicates the first Effective prior injection coefficients at time t. Indicates the first Time-based anomaly suppression gate, Indicates the first Prior injection coefficients at time step; Based on the effective prior injection coefficient, the current measurement state, and the current prior state, the current posterior state is repaired to obtain the target posterior state: ; in, Indicates the first The posterior state of the target at time t.

[0012] Optionally, the action estimation method for a task with sufficient state under missing sensor data, wherein the step of limiting the magnitude of the target posterior state and predicting the activity type of the target object using the target posterior state within a preset range specifically includes: Based on the aforementioned constraints, the magnitude between the target posterior state and the current measurement state is limited: ; in, express and The norm between, express and The norm between; Based on the task output function, determine whether the target posterior state satisfies the following conditions. continuous: ; in, express The task output, express The task output, Indicates that the posterior state of the target satisfies continuous, Indicates in the task output function The Lipschitz constant is given below; Targeting the satisfaction For all consecutive task categories, each category interval is defined as the task confidence level corresponding to that task category: ; in, Indicates the classification interval, Indicates the first The largest category predicted by the current measured state at time step. express In category The output logic on, express In category The output logic on, The set of categories representing the current measurement state; When the classification interval or the task confidence meets a preset condition, the target posterior state corresponding to the task category is within a reliable range: ; ; in, Indicates in the task output function The Lipschitz constant is given below; The target's posterior state is input into the action prediction model to predict the action of the target object at the next moment.

[0013] Optionally, the action estimation method for a task-sufficient state under missing sensor data, wherein the step of performing task state anomaly detection on the current posterior state based on multiple detection indicators of each of the human body sensors, repairing the current posterior state with the prior injection coefficient to obtain a target posterior state, and limiting the amplitude of the target posterior state, predicting the activity type of the target object using the target posterior state within a preset range, further includes: The multidimensional cost vector of each of the human body sensors is constructed into a scalar comprehensive cost: ; in, Indicates the first Scalar overall cost of personal body sensors Indicates the first Multidimensional cost vector of personal body sensors express The weight parameters, Indicates transpose; We construct the total loss function by adding dynamic consistency loss, measurement-state task loss, task sufficiency constraint, reliability constraint, measurement hold constraint, cost constraint, anomaly suppression related terms, and stability constraint. ; in, Represents the total loss function. This indicates losses in downstream tasks. All represent weighting coefficients. Indicates from time 1 to time 2. Prior state at time The task output, Represents dynamic posterior state conditions. Represents a dynamic operator. Indicates the first The current measurement state at any given time. Indicates the first The posterior state of the target at time t. express and The square of the norm between Indicates from time 1 to time 2. Measurement state at time The task output, This indicates a loss of task adequacy. Indicates the number of human body sensors. Indicates the first Personal body sensors in the first The problem of choosing a time is represented as follows: Indicates the first Posterior state at time step and The norm between, This represents the maximum limit parameter. Indicates the first Prior state at time and The norm between, Represents dynamic operators spectral radius, Indicates the preset stability threshold; ; in, Indicates based on posterior state Task risks, This indicates the mission risk under complete observational reference conditions. This indicates a pre-set acceptable risk level; The motion prediction model is trained using the total loss function, and multiple human sensors for data collection in the next round are determined using the constraints of all the human sensor selections.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an action estimation program for a task sufficiency state under missing sensor data stored in the memory and executable on the processor, wherein when the action estimation program for a task sufficiency state under missing sensor data is executed by the processor, it implements the steps of the action estimation method for a task sufficiency state under missing sensor data as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an action estimation program for a task sufficient state under missing sensor data, and when the action estimation program for a task sufficient state under missing sensor data is executed by a processor, it implements the steps of the action estimation method for a task sufficient state under missing sensor data as described above.

[0016] In this invention, multiple usable observation data collected by multiple human sensors on a target object are acquired. Each usable observation data is encoded into a latent space to obtain a corresponding latent representation. All latent representations are then fused to obtain the current measurement state. The historical posterior state under the acquired task-sufficient state is used to predict the current dynamic trend of the target object based on the historical posterior state and dynamic operators introduced into the latent space, resulting in the current prior state. A constraint optimization function for the current posterior state is constructed using the current measurement state and the current prior state. Prior injection coefficients are constructed based on the constraint optimization function, and the current posterior state is generated based on the prior injection coefficients. Based on multiple detection indicators of each human sensor, task state anomaly detection is performed on the current posterior state. The current posterior state is then repaired using the prior injection coefficients to obtain the target posterior state. The target posterior state is then amplitude-limited, and the activity type of the target object is predicted using the target posterior state within a preset range. This invention can obtain a more stable state representation for downstream identification, classification, or prediction tasks even when sensors are missing, observations are weak, sensor costs are limited, or observations are unstable. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the action estimation method for a task in a sufficient state under missing sensor data according to the present invention; Figure 2 This is a schematic diagram of part of the observed human sensor inputs in a preferred embodiment of the action estimation method for a task with sufficient data under missing sensor data of the present invention. Figure 3 This is a schematic diagram illustrating the construction of the latent state of a task-sufficient state, which is a preferred embodiment of the action estimation method for a task-sufficient state under missing sensor data of the present invention. Figure 4 This is a schematic diagram illustrating the state update while retaining measurement information, representing a preferred embodiment of the action estimation method for a task in a fully functional state under missing sensor data according to the present invention. Figure 5 This is a schematic diagram illustrating cost-aware observation selection and task prediction in a preferred embodiment of the action estimation method for a task in a sufficient state with missing sensor data according to the present invention. Figure 6 This is a schematic diagram of the training objective and closed-loop optimization mechanism of a preferred embodiment of the action estimation method for a task with sufficient state under missing sensor data of the present invention. Figure 7 This is a comparison chart of the task performance and calibration performance of different methods under limited observation conditions of a preferred embodiment of the action estimation method for a task with sufficient state but missing sensor data according to the present invention. Figure 8 This is a performance-cost frontier comparison diagram of a preferred embodiment of the action estimation method for a task with sufficient state under missing sensor data in this invention. Figure 9 This is a diagnostic diagram of the missing observation robustness and strong observation protection mechanism of a preferred embodiment of the action estimation method for a task with sufficient state under missing sensor data of the present invention. Figure 10 This is a structural diagram of a preferred embodiment of the action estimation system for a task with sufficient state under missing sensor data according to the present invention; Figure 11 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This invention provides a method and system for estimating the sufficient state of a task under missing observations in wearable multi-sensor human motion recognition and health status monitoring scenarios. It is applicable to situations where multiple sensing devices, such as wearable terminals, smartphones, smart bracelets, inertial measurement units, heart rate sensors, and respiration sensors, work collaboratively. In practical use, due to factors such as some sensors being disconnected, changes in wearing position, limited sampling costs, abnormal wireless transmission, or increased local observation noise, the system often cannot continuously obtain complete multi-sensor observation data. Therefore, this invention does not aim to completely restore all original sensor channels, but rather to obtain a sufficient state of the task that can support human motion recognition, motion state estimation, abnormal event detection, or health status monitoring. In the embodiments disclosed in this invention, human motion recognition is used as an example to describe in detail the motion estimation method for the sufficient state of the task under missing sensor data.

[0020] The preferred embodiment of the present invention describes a method for estimating the action of a task in a sufficient state with missing sensor data, such as... Figure 1 As shown, the action estimation method for a task with sufficient state under missing sensor data includes the following steps: Step S10: Acquire multiple available observation data collected by multiple human body sensors on the target object, encode each available observation data into the latent space to obtain the corresponding latent representation, and fuse all the latent representations to obtain the current measurement state.

[0021] Specifically, based on the acquisition command of the previous moment, multiple human body sensors are controlled to collect multiple available observation data of the target object in the target task, and a corresponding availability mask is added to each type of available observation data; Each available observation is input into the corresponding branch encoder for feature extraction, thereby encoding each available observation into the latent space to obtain the corresponding latent representation: ; in, Indicates the first The available observation data in the first The potential representation of time, Indicates the first One branch encoder, Indicates the first Personal body sensors in the first Available observational data at time, This represents the set of sensor group numbers that currently provide available observation data; Each latent representation is decomposed into a shared task state and a private residual state. Based on all the shared task states and all the private residual states, all the latent representations are fused using a measurement state fusion function to obtain the current measurement state. ; in, Indicates the first Time of the first A shared task state between individual human sensors and downstream tasks is used to participate in cross-sensor fusion and dynamic prediction. Indicates the first Time of the first Private residual states in individual human sensors are used to preserve information unique to the sensor group itself. ; in, Indicates the current measurement status. Represents the measurement-state fusion function. Indicates availability mask.

[0022] Among them, such as Figure 2 The system first acquires multi-sensor sequences, sensor availability masks, sensor acquisition cost information, and historical posterior states from different wearing positions or different types of devices on the human body. These data are not the acquisition data of all human body sensors, but rather the acquisition data of human body sensors needed in the current recognition round based on the previous detection. Under the condition of incomplete sensor observation, the system uses available signals, missing states, and acquisition costs to estimate subsequent states, which can save prediction costs and improve computational efficiency.

[0023] Specifically, at the current moment, based on the sensor online status, data reception status, communication status, and observation quality detection results, the set of currently available sensors is obtained. and the corresponding availability mask If the first If the data collected by the personal body sensor is valid, then: ; Conversely, it exists: .

[0024] Furthermore, for each human sensor, the available observation data is transmitted to the corresponding branch encoder for feature extraction to obtain the latent representation of the sensor group. To simultaneously retain shared task information across sensors and information specific to each individual sensor, the latent representation is further decomposed into a shared task state (used for cross-sensor fusion and Koopman dynamic prediction) and a private residual state (used to retain information specific to the sensor group). Based on the available sensor set and availability mask, the encoding results of the currently available sensors can be fused to obtain the current measurement state. In the embodiments disclosed in this invention, the goal is not to first complete all missing sensor data, nor to simply stitch together all sensor data. Instead, under conditions of missing observations, only the currently available sensors are used to form a measurement state capable of discriminating against downstream tasks. This avoids interference from missing or low-quality channels on state estimation and provides a stable task-related state foundation for subsequent Koopman dynamic prediction, observation reliability assessment, and reliability-constrained posterior fusion.

[0025] Step S20: Obtain the historical posterior state under the sufficient state of the task, and predict the current dynamic trend of the target object based on the historical posterior state and the dynamic operators introduced into the potential space to obtain the current prior state.

[0026] Among them, the task-sufficient state refers to the posterior state that does not require the complete recovery of all the original observations of the missing sensors, but rather requires that the state's ability to discriminate downstream tasks is close to that under the condition of complete observation; downstream tasks may include human motion recognition, abnormal event detection, motion state estimation, or health status monitoring.

[0027] Specifically, obtain the complete observation history at the current moment, and use the complete observation history to calculate the risk of the first task: ; in, Indicates the first The primary task and risk at all times This represents a function indicating the risk of a task. This represents the task loss function. This represents the output function of a reference mission based on a complete observation history. Indicates up to the number The complete observation history of the moment, Indicates the downstream task label; Obtain the predicted dynamic posterior state from the previous time step, and construct the second task risk based on the dynamic posterior state: ; in, Indicates the first The second task risk at any given moment This represents the task output function based on the posterior state. Indicates the dynamic posterior state; Based on the first task risk and the second task risk, determine whether the dynamic posterior state is a historical posterior state of a task-sufficient state: If the difference between the first task risk and the second task risk is not greater than a first non-negative threshold, or the conditional mutual information between the first task risk and the second task risk is not greater than a second non-negative threshold, then the dynamic posterior state is determined to be a historical posterior state of a task-sufficient state. ; ; in, This represents the first non-negative threshold. Conditional mutual information refers to mutual information under dynamic posterior state conditions. and The amount of information shared between them Indicates the second non-negative threshold; A dynamic operator is introduced into the latent space. Based on the historical posterior state and the dynamic operator, the current dynamic trend of the target object is predicted to obtain the current prior state at the current moment. ; in, Indicates the first The current prior state at any given moment. This represents a dynamic operator.

[0028] Among them, such as Figure 3 As shown, based on the predicted posterior state of the previous moment, the dynamic trend at that time is obtained by using Koopman dynamic prediction, thus obtaining the current prior state; however, the present invention needs to further determine whether the estimated posterior state is "sufficient" and whether it can accurately predict the target object's action recognition in the next moment.

[0029] This process can compare the estimated posterior state's risk in the downstream task with the task risk under the complete observation reference condition. If the difference between the first task risk and the second task risk is not greater than a first non-negative threshold, then the estimated posterior state at the previous time step can be determined to meet the conditions. Alternatively, from the perspective of information preservation, if the conditional mutual information between the first task risk and the second task risk is not greater than a second non-negative threshold, then the complete observation history is determined. Task tags The amount of additional information that can be provided is relatively small, and the posterior state has already preserved enough task-related information.

[0030] To utilize the dynamic evolution information in historical states, this invention introduces the Koopman dynamic operator into the task's latent space. It predicts the current dynamic trend and obtains the current prior state at the current moment.

[0031] This invention does not pursue the accurate restoration of every original channel of the missing sensor, but directly constrains whether the estimated state is useful enough for classification, identification, prediction or health monitoring tasks. Compared with traditional missing data imputation methods, this approach can reduce the noise and computational overhead caused by task-irrelevant reconstruction and make the state estimation results more directly serve downstream tasks.

[0032] Furthermore, a normalized dynamic innovation magnitude is constructed based on the current measurement state and the current prior state to measure the consistency between the current measurement state and the current prior state. ; ; in, Indicates the first The dynamic innovation vector between the current measured state and the current prior state at time step [time]. Indicates the first The current measurement state at any given time. Indicates the first The current prior state at any given moment. Indicates the first The normalized dynamic innovation magnitude between the current measured state and the current prior state at time t. Indicates the first The potential state dimension at any given moment; To measure the consistency between the current measurement state and the dynamic prior state, a dynamic innovation vector can be further constructed and defined as the normalized dynamic innovation magnitude. If the current measurement state and the current prior state are close, it indicates that the current observation is consistent with the historical dynamics. If the two are significantly different, there may be situations such as observational anomalies, sensor misalignment, sudden state changes, or unreliable dynamic priors.

[0033] The reliability of the current measurement state is constructed based on the normalized dynamic innovation magnitude, observation noise, historical consistency, task confidence, outlier observation score, and domain offset score. ; in, Indicates the first Reliability at any moment This represents the activation function. All of these represent learnable parameters. Indicates the first The current effective missing rate at any given time. Indicates the first Observation noise at any given moment Indicates the first Historical consistency of time Indicates the first Task confidence at any given time Indicates the first The score of abnormal observations at any given time. Indicates the first Domain offset or sensor misalignment fraction at any given time; ; ; in, Indicates the first The largest category predicted by the current measured state at time step. express In category The output logic on, express In category The output logic on, The set of categories representing the current measurement state.

[0034] Furthermore, this invention comprehensively judges the reliability of the current measurement state based on multiple factors, including dynamic innovation magnitude, observation noise, historical consistency, task confidence, outlier observation score, and domain offset score; among which the missing rate... Higher, dynamic innovation range Larger, observation noise Larger, abnormal observation score The higher the domain offset score The higher the value, the less reliable the current measurement status; historical consistency The higher the task confidence level The higher the value, the more reliable the current measurement status.

[0035] Step S30: Construct a constraint optimization function for the current posterior state using the current measurement state and the current prior state, construct prior injection coefficients based on the constraint optimization function, and generate the current posterior state based on the prior injection coefficients.

[0036] Among them, such as Figure 4 As shown, it is also necessary to continue to consider the extent of prior use and determine whether more measurement states or more prior states are needed for assistance.

[0037] Specifically, the constrained optimization function for the current posterior state is constructed using the current measurement state and the current prior state: ; in, Indicates the first The current posterior state at time t. Indicates the first The measurement state weights at time t. This represents the optimized posterior state. Indicates the first The current measurement state at any given time. Indicates the first Dynamic prior weights at time points, express and The square of the norm between express and the current prior state The square of the norm between; Based on the constraint optimization function, determine the constraint conditions: ; ; ; in, This represents the maximum limit parameter. express and The norm between, express and when The norm between, Indicates the first Reliability parameters at any given time. This indicates an extremely small positive number that prevents the denominator from being zero; A prior injection ratio is constructed based on the measured state weights and the dynamic prior weights, and prior injection coefficients are constructed based on the prior injection ratios: ; ; in, Indicates the first Prior injection ratio at time, Indicates the first Prior injection coefficients at time step, Represents the shearing function; Using the current measurement state as the anchor point, construct the current posterior state based on the prior injection coefficients: .

[0038] In this invention, posterior state estimation is formulated as a reliability-weighted constrained optimization problem, where the posterior state... The current measurement status cannot be completely ignored. Nor can it be influenced by dynamic prior states when observations are reliable. Forcibly pulled to the side.

[0039] After defining the constraint optimization function, the constraint conditions can be further determined, in which reliability is considered. The larger the value, the more reliable the measurement. The larger the measurement state weight, the greater the weight; conversely, the smaller the weight, the greater the dynamic prior weight. The larger.

[0040] Without considering constraints, the prior injection ratio corresponding to the reliability-weighted fusion can be determined, thereby constructing the prior injection coefficients. This allows the posterior state to be written in a form anchored to the measured state. In this case, the prior injection coefficients... It can represent dynamic prior states. For the current measurement status The degree of influence; at this point, it can be determined that, when the measurement is reliable, Larger Smaller Smaller, posterior state Closer to the measurement state When measurements are severely missing or unreliable, Smaller Larger Increase, posterior state Make more use of dynamic prior states Compensation is performed; this process is not a simple averaging of the measurement state and dynamic prior state, but rather an adaptive determination of how much dynamic prior can be injected based on the current observation reliability, and the use of constraints to prevent reliable measurements from being corrupted by inaccurate priors.

[0041] Step S40: Based on multiple detection indicators of each of the human body sensors, perform task state anomaly detection on the current posterior state, combine the prior injection coefficient to repair the current posterior state to obtain the target posterior state, and limit the amplitude of the target posterior state. Use the target posterior state within a preset range to predict the activity type of the target object.

[0042] In this invention, a cost-constrained observation selection mechanism is introduced during the state estimation process to determine which sensor groups should be prioritized for acquisition in the next moment. Unlike active sensing methods that only consider a single cost or uncertainty, this invention simultaneously considers task benefits, dynamic innovation benefits, sensor redundancy relationships, and multi-dimensional acquisition costs.

[0043] Specifically, the task benefit, dynamic innovation benefit, redundancy penalty, and multidimensional cost vector of each human sensor are obtained. For each human sensor, a selection score is constructed based on the task benefit, dynamic innovation benefit, redundancy penalty, and multidimensional cost vector. ; in, Indicates the first Personal body sensors in the first The choice of time to score points They represent The weight, Indicates the first Personal body sensors in the first Real-time task rewards Indicates the first Personal body sensors in the first Dynamic innovation benefits at all times Indicates the first Personal body sensors in the first Redundancy penalty in time Indicates the first Personal body sensors in the first The multidimensional cost vector at time step. Indicates transpose; ; ; in, This indicates uncertainty regarding the task. This represents the expected decrease in mission uncertainty. Indicates the first Personal body sensors in the first The expected dynamic innovation magnitude at any given moment; ; in, They represent the first The energy cost, latency cost, bandwidth cost, privacy cost, and computing cost of personal body sensors; Based on all the human body sensors, a sensor relationship graph is constructed to obtain the relationship strength between each of the human body sensors: ; in, Indicates the first Time of the first Personal body sensors and the The strength of the relationship between individual human sensors This represents the activation function. express The weighted value, Indicates the first Time of the first Personal body sensors and the Differences in measurement states between individual human body sensors express The weighted value, Indicates from the first Time to the The first moment Personal body sensors and the Differences in dynamic innovation vectors among individual human sensors; The acquisition process of the human body sensor is monitored based on the sensor relationship diagram, and a selection problem representation is constructed under multidimensional budget conditions based on all the selection scores, thus constructing constraints for the selection of the human body sensor: ; ; in, Indicates the first Time constraints, Indicates the first Personal body sensors in the first The problem of choosing a time is represented as follows: Indicates the number of human body sensors. Indicates the first Categories of personal body sensors Indicates multidimensional budgeting, Indicates the constraints for selecting human body sensors; Specifically, for each human sensor, its selection score can be calculated at the current moment based on its task benefit, dynamic innovation benefit, redundancy penalty, and multidimensional cost vector. The multidimensional cost vector can be determined based on the sensor's energy consumption cost, latency cost, bandwidth cost, privacy cost, and computational cost. The selection score of each human sensor can be used to determine whether to select that human sensor in the next moment. For example, if the selection score of the ankle inertial sensor is relatively low, it means that the data collected by the ankle inertial sensor in the next moment will not contribute much to action recognition. Therefore, it is not necessary to perform action recognition based on the data of the ankle inertial sensor, thereby controlling the ankle inertial sensor to stop data collection in the next moment. This can significantly reduce the interference of redundant data.

[0044] Furthermore, to avoid repeatedly acquiring highly similar or redundant sensors, this invention constructs a sensor relationship graph and defines the relationship strength between every two human sensors. This allows for the calculation of the redundancy penalty for each human sensor relative to the already selected sensor set, thus incorporating it into the factors considered when evaluating human sensors. Under the constraint of multidimensional budgeting, the sensor selection problem can be expressed in a constrained manner and determined based on the corresponding constraints. By selecting the sensor set that is most useful for the task, most useful for dynamic compensation, does not overlap with the selected sensors, and is cost-effective, the performance of task recognition and state estimation can be improved under constraints of limited energy consumption, limited bandwidth, limited latency, or privacy.

[0045] An anomaly suppression gate is constructed for each time step, and the anomaly suppression gate is introduced into the corresponding prior injection coefficient to obtain the effective prior injection coefficient: ; in, Indicates the first Effective prior injection coefficients at time t. Indicates the first Time-based anomaly suppression gate, Indicates the first Prior injection coefficients at time step; Based on the effective prior injection coefficient, the current measurement state, and the current prior state, the current posterior state is repaired to obtain the target posterior state: ; in, Indicates the first The posterior state of the target at time t.

[0046] In real-world sensor deployments, observation problems include not only sensor absence but also sensor misalignment, drift, sudden noise changes, asynchronous sampling, and channel contamination. To address these issues, this invention further introduces an anomaly observation suppression gate to adjust the impact of anomalous observations or anomalous priors on the posterior state.

[0047] After introducing an anomaly suppression gate, the prior injection coefficients can be optimized to obtain effective prior injection coefficients. Then, posterior fusion is performed. When observational anomalies, large domain shifts, or severe conflicts between priors and measurements are detected, the anomaly suppression gate is activated. This reduces the impact of unreliable information on the posterior state.

[0048] This invention not only addresses sensor missingness but also covers real-world deployment issues such as sensor misalignment, drift, sudden noise changes, asynchronous sampling, and channel contamination. (Anomaly suppression gate) It acts as a safety switch to reduce the interference of unreliable observations or unreliable priors on the posterior state under abnormal conditions.

[0049] Specifically, based on the constraints, the magnitude between the target posterior state and the current measurement state is limited: ; in, express and The norm between, express and The norm between; Based on the task output function, determine whether the target posterior state satisfies the following conditions. continuous: ; in, express The task output, express The task output, Indicates that the posterior state of the target satisfies continuous, Indicates in the task output function The Lipschitz constant is given below; Targeting the satisfaction For all consecutive task categories, each category interval is defined as the task confidence level corresponding to that task category: ; in, Indicates the classification interval, Indicates the first The largest category predicted by the current measured state at time step. express In category The output logic on, express In category The output logic on, The set of categories representing the current measurement state; When the classification interval or the task confidence meets a preset condition, the target posterior state corresponding to the task category is within a reliable range: ; ; in, Indicates in the task output function The Lipschitz constant is given below; The target's posterior state is input into the action prediction model to predict the action of the target object at the next moment.

[0050] To prevent reliable observations from being compromised by inaccurate dynamic priors, this invention limits the magnitude of the deviation between the posterior state and the measurement state. If the magnitude between the target posterior state and the current measurement state can be limited to... Then the target posterior state Relative to the measurement state The offset will not exceed the dynamic prior state. With measurement status A fixed proportion of the difference between the two states ensures that, assuming the current measurement state is reliable, the posterior state will not be arbitrarily pulled away by the prior state, such as... Figure 5 As shown, action recognition can then be performed based on the target's posterior state.

[0051] Furthermore, if the task output function satisfy If the function is continuous (describing a finite rate of change in the function, a strong form of uniform continuity), then the variation in the task output is also limited, and the results are more stable. In this case, the classification interval of the measurement state will also meet the preset condition (i.e., ...). At this point, the classification confidence of the current measurement state is high enough that posterior fusion will not change the original classification result obtained from reliable measurement.

[0052] In this context, strong observation protection is not an abstract description, but rather a constraint on the range of posterior state movement through formulas. When the measurement state itself is already reliable, posterior fusion only allows dynamic prior compensation with a limited range, thereby improving the output stability of tasks such as human action recognition, abnormal event detection, and health status monitoring.

[0053] Furthermore, the multidimensional cost vector of each of the human body sensors is constructed as a scalar comprehensive cost: ; in, Indicates the first Scalar overall cost of personal body sensors Indicates the first Multidimensional cost vector of personal body sensors express The weight parameters, Indicates transpose; We construct the total loss function by adding dynamic consistency loss, measurement-state task loss, task sufficiency constraint, reliability constraint, measurement hold constraint, cost constraint, anomaly suppression related terms, and stability constraint. ; in, Represents the total loss function. This indicates losses in downstream tasks. All represent weighting coefficients. Indicates from time 1 to time 2. Prior state at time The task output, Represents dynamic posterior state conditions. Represents a dynamic operator. Indicates the first The current measurement state at any given time. Indicates the first The posterior state of the target at time t. express and The square of the norm between Indicates from time 1 to time 2. Measurement state at time The task output, This indicates a loss of task adequacy. Indicates the number of human body sensors. Indicates the first Personal body sensors in the first The problem of choosing a time is represented as follows: Indicates the first Posterior state at time step and The norm between, This represents the maximum limit parameter. Indicates the first Prior state at time and The norm between, Represents dynamic operators spectral radius, Indicates the preset stability threshold; ; in, Indicates based on posterior state Task risks, This indicates the mission risk under complete observational reference conditions. This indicates a pre-set acceptable risk level; The motion prediction model is trained using the total loss function, and multiple human sensors for data collection in the next round are determined using the constraints of all the human sensor selections.

[0054] Among them, the Koopman dynamic consistency loss is constructed during the Koopman dynamic prior prediction; the measurement state task loss and task sufficiency constraint are constructed during multi-sensor data encoding and the identification of task sufficiency state of posterior state, respectively; the reliability constraint and measurement hold constraint are constructed during the reliability assessment of the current measurement state and the constraint of posterior state; the cost constraint is constructed during the introduction of cost constraint observation selection mechanism in the state estimation process; and the anomaly suppression correlation term is constructed when adjusting the influence of abnormal observations or abnormal priors on posterior state.

[0055] Among them, such as Figure 6 As shown, the total loss function constructed in this invention requires not only accurate model classification during training, but also that the latent state conforms to the Koopman dynamic law, that the measurement state itself has task discrimination ability, that the posterior state satisfies task sufficiency, that the sensor selection meets cost constraints, and that the measurement maintenance constraint prevents the posterior state from deviating from reliable observations, and that the Koopman stability constraint prevents dynamic prediction divergence.

[0056] Furthermore, to verify the effectiveness of this invention under conditions of missing observations and cost constraints, experiments were conducted using multiple wearable sensor recognition datasets, including UCI-HAR, RealWorld HAR, REALDISP, and OPPORTUNITY. UCI-HAR is a dataset used to test recognition scenarios with high inertial sensor missing data; RealWorld HAR is a dataset used to test scenarios with multiple wearable sensors in different locations; REALDISP is a dataset used to test scenarios with missing body position sensors and displacement; and OPPORTUNITY is a dataset used to test whether posterior fusion would compromise reliable measurements when current observations are strong.

[0057] The experiment employs a sensor group-level missing protocol, rather than random occlusion of individual feature points. Random missing is used to simulate general sensor unavailability, block missing is used to simulate continuous disconnections or long-term missing periods, and the fixed strong observation protocol can test whether the posterior update can maintain the stability of the measurement state when the currently available sensors already have strong discriminative capabilities.

[0058] Comparison methods include measurement-only methods using only the current measurement state, fixed Koopman-Kalman posterior fusion methods, Prior-only Koopman methods (Kopman methods based on linear operator approximation), missing data imputation methods, arbitrary missing data routing methods, random budget selection, and entropy-based active sensing methods. Evaluation metrics include macro-average F1 score, lowest class recall, negative log-likelihood, expected calibration error, and normalized sensor array cost.

[0059] Furthermore, Table 1 presents the core performance results under limited observation conditions: Table 1: Comparison of Core Performance of Different Methods under Limited Observation Conditions

[0060] As shown in Table 1, in missing observation scenarios of UCI-HAR (first dataset), RealWorld HAR (second dataset), REALDISP (third dataset), and OPPORTUNITY (fourth dataset), this invention achieves higher macro-average F1 scores and the lowest class recall compared to methods such as Measurement-only (methods that only estimate observation states), Fixed Koopman-Kalman (Fixed KKF, a method that uses fixed Koopman-Kalman updates), and Best mask-router (optimal missing pattern routing method). At the same time, it has lower negative log-likelihood and expected calibration error, indicating that the posterior state obtained by this method not only has better task discrimination ability but also better confidence calibration performance.

[0061] Furthermore, such as Figure 7 As shown, on four datasets, the macro-average F1 score of this invention (e.g.) Figure 7 (a) and minimum category recall (e.g.) Figure 7 In terms of the two main task indicators (b) in the middle, the overall level is at or near the optimal level, while the negative log-likelihood (e.g.) is at a relatively low level. Figure 7 (c) and expected calibration error (e.g.) Figure 7 The accuracy is also lower in (d) of the data, which indicates that the present invention does not simply improve the average accuracy, but simultaneously improves class coverage, prediction stability and confidence reliability.

[0062] Especially on the REALDISP dataset, due to significant sensor displacement and missing data, the current measurement state is weak. This invention uses Koopman dynamic prior compensation to compensate for historical state information, which improves the macro-average F1 score from 0.706 for Measurement-only to 0.755 and the minimum recall from 0.548 to 0.641. This demonstrates that this invention can improve the quality of state estimation in scenarios with weak observations and sensor misalignment.

[0063] On the OPPORTUNITY dataset, currently available body-IMU observations already possess strong discriminative capabilities. Fixed Koopman-Kalman fusion pulls the posterior state towards an inaccurate prior, causing the macro-average F1 score to drop from 0.984 (measurement-only) to 0.952, and the minimum recall to drop from 0.955 to 0.827. In contrast, this invention maintains reliable measurements through measurement-preserving posterior updates, achieving a macro-average F1 score of 0.985 and a minimum recall of 0.963, demonstrating the effectiveness of the strong observation protection mechanism.

[0064] Furthermore, to verify the cost-aware observation selection mechanism of the present invention, the task performance under different sensor budgets was compared. The cost was represented by the normalized sensor group cost, with a full sensor acquisition cost of 1.0. Random budget selection only randomly selects a portion of the sensors; the entropy-based active sensing method mainly selects sensors based on task uncertainty; the present invention simultaneously considers task benefits, dynamic innovation benefits, sensor redundancy relationships, and acquisition costs.

[0065] Among them, such as Figure 8 As shown, under the same normalized sensor cost, the macro-average F1 value of the present invention (e.g.) Figure 8 (a) and minimum recall (e.g.) Figure 8 (b) shows that the proposed observation selection strategy outperforms both random budget selection and entropy-based active sensing methods. For example, in cost frontier experiments on the RealWorld HAR and REALDISP datasets, the proposed strategy achieves higher task performance at a normalized cost of approximately 0.4, demonstrating that it can achieve higher task benefits with less sensor overhead. This result indicates that more sensors are not necessarily better, nor is it sufficient to select only the sensors with the highest uncertainty. By introducing dynamic innovation benefits and redundancy penalties, the proposed strategy avoids repeatedly acquiring highly correlated sensor groups and prioritizes low-cost sensors that are more helpful to the current posterior state and downstream tasks, thereby improving resource utilization efficiency.

[0066] Furthermore, to further analyze the working mechanism of this invention, experiments were conducted to examine the changes in task performance under different missing proportions and the degree of prior injection under different measurement intensities. A higher missing proportion indicates a weaker current measurement state; a higher measurement intensity indicates a more reliable current measurement state. An ideal state estimation method should utilize dynamic prior compensation under weak observations and restrict prior injection under strong observations to avoid compromising reliable measurements.

[0067] Among them, such as Figure 9 (a) and Figure 9As shown in (b), as the missing proportion increases, the Measurement-only method degrades rapidly because it can only rely on currently available observations; the Prior-only method, while utilizing historical dynamics, tends to ignore current measurements; Fixed KKF can compensate for insufficient observations in some cases, but forced fusion can produce side effects in scenarios with strong observations. This invention maintains a higher macro-average F1 score and minimum recall rate under different missing proportions, indicating that the joint design of dynamic prior compensation and measurement protection can improve the robustness to missing observations.

[0068] And such Figure 9 As shown in (c), the reliability transition curve indicates that when the measurement intensity is weak, the prior injection coefficient is high, and the system utilizes Koopman dynamic priors more to compensate for insufficient current observations. When the measurement intensity increases, the prior injection coefficient and posterior offset gradually decrease, and the posterior state is closer to the measurement state. This result is consistent with the measurement-holding posterior update mechanism of this invention, demonstrating that this invention can adaptively switch between "weak observation compensation" and "strong observation protection" operating states based on observation reliability.

[0069] Furthermore, it should be noted that this invention can be used not only for wearable human activity recognition, but also for scenarios such as industrial equipment fault prediction, vehicle state estimation and localization, medical physiological signal monitoring, smart home event detection, mobile terminal multi-sensor recognition, and robot multi-sensor state estimation. This invention can obtain a more stable state representation for downstream recognition, classification, or prediction tasks even when sensors are missing, observations are weak, sensor costs are limited, or observations are unstable.

[0070] Furthermore, such as Figure 10 As shown, based on the above-mentioned action estimation method for a sufficient task state under missing sensor data, the present invention also provides a corresponding action estimation system for a sufficient task state under missing sensor data, wherein the action estimation system for a sufficient task state under missing sensor data includes: The data fusion module 51 is used to acquire multiple available observation data collected by multiple human body sensors on the target object, encode each of the available observation data into the latent space to obtain the corresponding latent representation, and fuse all the latent representations to obtain the current measurement state; Prediction module 52 is used to obtain the historical posterior state under the sufficient state of the task, and predict the current dynamic trend of the target object based on the historical posterior state and the dynamic operators introduced into the potential space to obtain the current prior state. The posterior state construction module 53 is used to construct a constraint optimization function for the current posterior state using the current measurement state and the current prior state, construct prior injection coefficients based on the constraint optimization function, and generate the current posterior state based on the prior injection coefficients. The behavior prediction module 54 is used to perform task state anomaly detection on the current posterior state based on multiple detection indicators of each of the human body sensors, repair the current posterior state by combining the prior injection coefficient to obtain the target posterior state, limit the amplitude of the target posterior state, and predict the activity type of the target object using the target posterior state within a preset range.

[0071] Furthermore, such as Figure 11 As shown, based on the above-mentioned action estimation method and system for a task with sufficient state under missing sensor data, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 11 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0072] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores an action estimation program 40 for a task-sufficient state under missing sensor data. This action estimation program 40 for a task-sufficient state under missing sensor data can be executed by the processor 10 to implement the action estimation method for a task-sufficient state under missing sensor data in this application.

[0073] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing an action estimation method for a task in a sufficient state under missing sensor data.

[0074] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0075] In one embodiment, when the processor 10 executes the action estimation program 40 for a sufficient state of task under missing sensor data in the memory 20, it implements the steps of the action estimation method for a sufficient state of task under missing sensor data as described above.

[0076] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an action estimation program for a task-sufficient state under missing sensor data, and the action estimation program for a task-sufficient state under missing sensor data, when executed by a processor, implements the steps of the action estimation method for a task-sufficient state under missing sensor data as described above.

[0077] In summary, this invention provides a method and related equipment for action estimation in a task-sufficient state with missing sensor data. The method includes: acquiring multiple available observation data collected by multiple human sensors on a target object; encoding each available observation data into a latent space to obtain a corresponding latent representation; fusing all latent representations to obtain a current measurement state; acquiring historical posterior states under the task-sufficient state; predicting the current dynamic trend of the target object based on the historical posterior states and dynamic operators introduced into the latent space to obtain a current prior state; constructing a constrained optimization function for the current posterior state using the current measurement state and the current prior state; constructing prior injection coefficients based on the constrained optimization function; generating the current posterior state based on the prior injection coefficients; performing task state anomaly detection on the current posterior state based on multiple detection indicators of each human sensor; repairing the current posterior state with the prior injection coefficients to obtain a target posterior state; limiting the amplitude of the target posterior state; and predicting the activity type of the target object using the target posterior state within a preset range. This invention can obtain a more stable state representation for downstream identification, classification, or prediction tasks when sensors are missing, observations are weak, sensor costs are limited, or observations are unstable.

[0078] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0079] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0080] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for action estimation of task sufficiency state under missing sensor data, characterized in that, The action estimation method for a task in a fully-sufficient state under missing sensor data includes: Multiple usable observation data collected by multiple human body sensors on the target object are acquired, each of the usable observation data is encoded into a latent space to obtain a corresponding latent representation, and all the latent representations are fused to obtain the current measurement state; The process of acquiring multiple usable observation data points collected by multiple human body sensors on the target object, encoding each usable observation data point into a latent space to obtain a corresponding latent representation, and fusing all the latent representations to obtain the current measurement state specifically includes: Based on the acquisition command from the previous moment, control multiple human body sensors to collect multiple available observation data of the target object in the target task, and add a corresponding availability mask for each type of available observation data; Each available observation is input into the corresponding branch encoder for feature extraction, thereby encoding each available observation into the latent space to obtain the corresponding latent representation: ； in, Indicates the first The available observation data in the first The potential representation of time, Indicates the first One branch encoder, Indicates the first Personal body sensors in the first Available observational data at time, This represents the set of sensor group numbers that currently provide available observation data; Each latent representation is decomposed into a shared task state and a private residual state. Based on all the shared task states and all the private residual states, all the latent representations are fused using a measurement state fusion function to obtain the current measurement state. ； in, Indicates the first Time of the first A shared task state between individual human sensors and downstream tasks is used to participate in cross-sensor fusion and dynamic prediction. Indicates the first Time of the first Private residual states in personal body sensors; ； in, Indicates the current measurement status. Represents the measurement-state fusion function. Indicates availability mask; The historical posterior states under the obtained task-sufficient state are used to predict the current dynamic trend of the target object based on the historical posterior states and the dynamic operators introduced into the potential space, thereby obtaining the current prior state. A constraint optimization function for the current posterior state is constructed using the current measurement state and the current prior state. Prior injection coefficients are constructed based on the constraint optimization function, and the current posterior state is generated based on the prior injection coefficients. Based on multiple detection indicators of each of the human body sensors, the current posterior state is subjected to task state anomaly detection. The current posterior state is repaired by combining the prior injection coefficient to obtain the target posterior state. The target posterior state is then subject to amplitude limitation. The target posterior state within a preset range is used to predict the activity type of the target object.

2. The action estimation method for a task with sufficient state under missing sensor data according to claim 1, characterized in that, The historical posterior states obtained under the sufficient task state are used to predict the current dynamic trend of the target object based on the historical posterior states and the dynamic operators introduced into the latent space, thereby obtaining the current prior state, specifically including: Obtain the complete observation history at the current moment, and use the complete observation history to calculate the risk of the first task: ； in, Indicates the first The primary task and risk at all times This represents a function indicating the risk of a task. This represents the task loss function. This represents the output function of a reference mission based on a complete observation history. Indicates up to the number The complete observation history of the moment, Indicates the downstream task label; Obtain the predicted dynamic posterior state from the previous time step, and construct the second task risk based on the dynamic posterior state: ； in, Indicates the first The second task risk at any given moment This represents the task output function based on the posterior state. Indicates the dynamic posterior state; Based on the first task risk and the second task risk, determine whether the dynamic posterior state is a historical posterior state of a task-sufficient state: If the difference between the first task risk and the second task risk is not greater than a first non-negative threshold, or the conditional mutual information between the first task risk and the second task risk is not greater than a second non-negative threshold, then the dynamic posterior state is determined to be a historical posterior state of a task-sufficient state. ； ； in, This represents the first non-negative threshold. Conditional mutual information refers to mutual information under dynamic posterior state conditions. and The amount of information shared between them Indicates the second non-negative threshold; A dynamic operator is introduced into the latent space. Based on the historical posterior state and the dynamic operator, the current dynamic trend of the target object is predicted to obtain the current prior state at the current moment. ； in, Indicates the first The current prior state at any given moment. This represents a dynamic operator.

3. The action estimation method for a task with sufficient state under missing sensor data according to claim 1, characterized in that, The process involves obtaining the historical posterior state under the acquired task-sufficient state, predicting the current dynamic trend of the target object based on the historical posterior state and the dynamic operators introduced into the latent space, to obtain the current prior state, and then further including: A normalized dynamic innovation magnitude is constructed based on the current measurement state and the current prior state to measure the consistency between the current measurement state and the current prior state: ； ； in, Indicates the first The dynamic innovation vector between the current measured state and the current prior state at time step [time]. Indicates the first The current measurement state at any given time. Indicates the first The current prior state at any given moment. Indicates the first The normalized dynamic innovation magnitude between the current measured state and the current prior state at time t. Indicates the first The potential state dimension at any given moment; The reliability of the current measurement state is constructed based on the normalized dynamic innovation magnitude, observation noise, historical consistency, task confidence, outlier observation score, and domain offset score. ； in, Indicates the first Reliability at any moment This represents the activation function. All of these represent learnable parameters. Indicates the first The current effective missing rate at any given time. Indicates the first Observation noise at any given moment Indicates the first Historical consistency of time Indicates the first Task confidence at any given time Indicates the first The score of abnormal observations at any given time. Indicates the first Domain offset or sensor misalignment fraction at any given time; ； ； in, Indicates the first The largest category predicted by the current measured state at time step. express In category The output logic on, express In category The output logic on, The set of categories representing the current measurement state.

4. The action estimation method for a task with sufficient state under missing sensor data according to claim 1, characterized in that, The step of constructing a constrained optimization function for the current posterior state using the current measurement state and the current prior state, constructing prior injection coefficients based on the constrained optimization function, and generating the current posterior state based on the prior injection coefficients specifically includes: Construct a constrained optimization function for the current posterior state using the current measurement state and the current prior state: ； in, Indicates the first The current posterior state at time t. Indicates the first The measurement state weights at time t. This represents the optimized posterior state. Indicates the first The current measurement state at any given time. Indicates the first Dynamic prior weights at time points, express and The square of the norm between express and the current prior state The square of the norm between; Based on the constraint optimization function, determine the constraint conditions: ； ； ； in, This represents the maximum limit parameter. express and The norm between, express and when The norm between, Indicates the first Reliability parameters at any given time. This indicates an extremely small positive number that prevents the denominator from being zero; A prior injection ratio is constructed based on the measured state weights and the dynamic prior weights, and prior injection coefficients are constructed based on the prior injection ratios: ； ； in, Indicates the first Prior injection ratio at time, Indicates the first Prior injection coefficients at time step, Represents the shearing function; Using the current measurement state as the anchor point, construct the current posterior state based on the prior injection coefficients: 。 5. The action estimation method for a task with sufficient state under missing sensor data according to claim 4, characterized in that, The process of detecting task state anomalies in the current posterior state based on multiple detection indicators of each of the human body sensors, and then repairing the current posterior state using the prior injection coefficients to obtain the target posterior state, specifically includes: Obtain the task reward, dynamic innovation reward, redundancy penalty, and multidimensional cost vector for each human sensor. For each human sensor, construct a selection score based on the task reward, dynamic innovation reward, redundancy penalty, and multidimensional cost vector. ； in, Indicates the first Personal body sensors in the first The choice of time to score points They represent The weight, Indicates the first Personal body sensors in the first Real-time task rewards Indicates the first Personal body sensors in the first Dynamic innovation benefits at all times Indicates the first Personal body sensors in the first Redundancy penalty in time Indicates the first Personal body sensors in the first The multidimensional cost vector at time step. Indicates transpose; ； ； in, This indicates uncertainty regarding the task. This represents the expected decrease in mission uncertainty. Indicates the first Personal body sensors in the first The expected dynamic innovation magnitude at any given moment; ； in, They represent the first The energy cost, latency cost, bandwidth cost, privacy cost, and computing cost of personal body sensors; Based on all the human body sensors, a sensor relationship graph is constructed to obtain the relationship strength between each of the human body sensors: ； in, Indicates the first Time of the first Personal body sensors and the The strength of the relationship between individual human sensors This represents the activation function. express The weighted value, Indicates the first Time of the first Personal body sensors and the Differences in measurement states between individual human body sensors express The weighted value, Indicates from the first Time to the The first moment Personal body sensors and the Differences in dynamic innovation vectors among individual human sensors; The acquisition process of the human body sensor is monitored based on the sensor relationship diagram, and a selection problem representation is constructed under multidimensional budget conditions based on all the selection scores, thus constructing constraints for the selection of the human body sensor: ； ； in, Indicates the first Time constraints, Indicates the first Personal body sensors in the first The problem of choosing a time is represented as follows: Indicates the number of human body sensors. Indicates the first Categories of personal body sensors Indicates multidimensional budgeting, Indicates the constraints for selecting human body sensors; An anomaly suppression gate is constructed for each time step, and the anomaly suppression gate is introduced into the corresponding prior injection coefficient to obtain the effective prior injection coefficient: ； in, Indicates the first Effective prior injection coefficients at time 10:00 Indicates the first Time-based anomaly suppression gate, Indicates the first Prior injection coefficients at time step; Based on the effective prior injection coefficient, the current measurement state, and the current prior state, the current posterior state is repaired to obtain the target posterior state: ； wherein, represents the target posterior state at time t.

6. The method of action estimation for task sufficiency state in absence of sensory data of claim 5, wherein, The step of limiting the magnitude of the target posterior state and predicting the activity type of the target object using the target posterior state within a preset range specifically includes: Based on the aforementioned constraints, the magnitude between the target posterior state and the current measurement state is limited: ； in, express and The norm between, express and The norm between; Based on the task output function, determine whether the target posterior state satisfies the following conditions. continuous: ； in, express The task output, express The task output, Indicates that the posterior state of the target satisfies continuous, Indicates in the task output function The Lipschitz constant is given below; In response to satisfy For all consecutive task categories, each category interval is defined as the task confidence level corresponding to that task category: ； in, Indicates the classification interval, Indicates the first The largest category predicted by the current measured state at time step. express In category The output logic on, express In category The output logic on, The set of categories representing the current measurement state; When the classification interval or the task confidence meets a preset condition, the target posterior state corresponding to the task category is within a reliable range: ； ； in, Indicates in the task output function The Lipschitz constant is given below; The target's posterior state is input into the action prediction model to predict the action of the target object at the next moment.

7. The method of action estimation for task sufficiency state in absence of sensory data of claim 1, wherein, The process involves detecting task state anomalies in the current posterior state based on multiple detection indicators of each human sensor, repairing the current posterior state using the prior injection coefficients to obtain the target posterior state, limiting the amplitude of the target posterior state, and predicting the activity type of the target object using the target posterior state within a preset range. The process further includes: The multidimensional cost vector of each of the human body sensors is constructed into a scalar comprehensive cost: ； in, Indicates the first Scalar overall cost of personal body sensors Indicates the first Multidimensional cost vector of personal body sensors express The weight parameters, Indicates transpose; We construct the total loss function by adding dynamic consistency loss, measurement-state task loss, task sufficiency constraint, reliability constraint, measurement hold constraint, cost constraint, anomaly suppression related terms, and stability constraint. ； in, Represents the total loss function. This indicates losses in downstream tasks. All represent weighting coefficients. Indicates from time 1 to time 2. Prior state at time The task output, Represents dynamic posterior state conditions. Represents a dynamic operator. Indicates the first The current measurement state at any given time. Indicates the first The posterior state of the target at time t. express and The square of the norm between Indicates from time 1 to time 2. Measurement state at time The task output, This indicates a loss of task adequacy. Indicates the number of human body sensors. Indicates the first Personal body sensors in the first The problem of choosing a time is represented as follows: Indicates the first Posterior state at time step and The norm between This represents the maximum limit parameter. Indicates the first Prior state at time and The norm between Represents dynamic operators spectral radius, Indicates the preset stability threshold; ； in, Indicates based on posterior state Task risks, This indicates the mission risk under complete observational reference conditions. This indicates a pre-set acceptable risk level; The motion prediction model is trained using the total loss function, and multiple human sensors for data collection in the next round are determined using the constraints of all the human sensor selections.

8. A terminal, characterized by comprising: The terminal includes: a memory, a processor, and an action estimation program for a task-sufficient state with missing sensor data stored in the memory and executable on the processor. When the action estimation program for a task-sufficient state with missing sensor data is executed by the processor, it implements the steps of the action estimation method for a task-sufficient state with missing sensor data as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an action estimation program for a task-sufficient state with missing sensor data, which, when executed by a processor, implements the steps of the action estimation method for a task-sufficient state with missing sensor data as described in any one of claims 1-7.

Images