Electronic component interaction control method and device, electronic equipment and storage medium

Abstract

The invention provides an electronic component interaction control method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting the interaction data of an electronic component from a plurality of sensors, and fusing the multi-modal feature vectors into a comprehensive feature vector through a weighted splicing fusion algorithm; constructing a modal relation mapping network, using an adaptive fusion network to combine the features of different modals, and using a deep self-attention mechanism to capture the relation between the modals; constructing a user behavior causal inference model, and inputting the user intention vector y into the user behavior causal inference model to infer potential behaviors of the user; constructing an inter-device collaboration graph model, adjusting resource allocation between devices, and outputting a resource allocation strategy; constructing an equipment anomaly detection model; and building a reinforcement learning model to dynamically adjust model parameters of the resource allocation strategy ropt, and dynamically adjusting environment changes and user requirements. According to the invention, comprehensive upgrading of interaction control of electronic component equipment is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of interactive control, and in particular relates to an electronic component interactive control method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, electronic devices have been deeply integrated into people's daily lives. Various electronic products such as smart phones, tablets, smart home devices, wearable devices, etc. have emerged in an endless stream, greatly changing people's working methods and living habits. Traditional electronic device interactive control methods mainly rely on single interactive methods such as fixed hardware buttons and touch screens. Although these methods can realize some basic control functions, they are unable to cope with the increasingly diverse needs and complex usage scenarios of users. In recent years, new interactive methods such as voice control and gesture control have been widely used, but these new methods still have many limitations. For example, the recognition rate of voice control systems in noisy environments drops significantly, and the accuracy and response speed of gesture control are limited by the existing sensor accuracy and algorithm performance.

[0003] There are some other key problems with existing interactive control technologies. First, the interactive systems of most electronic devices are based on predefined rules or static models. These systems lack adaptability and cannot adjust their own strategies according to changes in user behavior and the dynamics of the environment, resulting in a poor user experience. Secondly, the existing systems have limited ability to predict and generate feedback on user behavior, and cannot truly understand the complex needs and changing behaviors of users. In this case, users often need to perform multiple operations to achieve the desired effect, resulting in an unsmooth operating experience. In addition, existing multi-device collaborative control methods are mostly based on simple logical linkages and lack global optimization strategies, resulting in the inability to effectively collaborate between multiple devices and low efficiency of the overall system.

[0004] At the same time, with the increase of sensors in electronic devices and the popularization of multimodal data input, the amount of data is growing exponentially. Traditional data processing methods are difficult to quickly process and analyze a large amount of user interaction data while ensuring low latency, especially when it comes to privacy-sensitive scenarios. Data collection and processing also face huge security risks. The shortcomings of existing technologies in data privacy protection cannot meet the growing demand of users for privacy protection. In addition, existing device control technologies lack intelligent scheduling of device resources. Especially when processing multiple tasks, existing systems are difficult to efficiently allocate hardware resources such as processors and memory, resulting in low energy efficiency of devices and poor user experience. The above problems show that the existing electronic device interactive control technology has obvious limitations and cannot fully meet the needs of modern users for intelligence, personalization, high efficiency and data privacy protection. Therefore, a more advanced interactive control method and system are needed to overcome these shortcomings of the existing technology. Summary of the invention

[0005] The purpose of the present invention is to design an electronic component interactive control method, device, electronic device and storage medium, which combines a variety of advanced artificial intelligence technologies such as few-sample learning, generative adversarial network (GAN), causal reasoning, graph neural network (GNN), federated learning, quantum machine learning, etc., aiming to achieve a comprehensive upgrade of the interactive control of electronic devices.

[0006] In order to achieve the above object, a first aspect of the present invention provides an electronic component interactive control method, the method comprising the following steps:

[0007] S1. Collect interaction data of electronic components from multiple sensors, preprocess the data, obtain preprocessed data of multiple sensors, use weighted splicing fusion algorithm for data from different sensors to fuse multimodal feature vectors into a comprehensive feature vector, and output it;

[0008] S2. Construct a modal relationship mapping network to capture the nonlinear relationship between each modal feature in the comprehensive feature vector and introduce a feature regularization term By constraining the ratio of each modality feature, an adaptive fusion network is used to combine the features of different modalities and a deep self-attention mechanism is used to capture the relationship between modalities, outputting the user intention vector y;

[0009] S3. Build a user behavior causal inference model, input the user intention vector y into the user behavior causal inference model to infer the user's potential behavior, and use the deep learning-based adaptive feedback generation network to generate personalized feedback f according to the user's potential behavior. * The user behavior causal inference model uses the risk ratio to calculate the change in the probability of behavior under different intentions, and introduces a behavior diversity regularization term into the user behavior causal inference model. constraining the predicted diversity of potential behaviors;

[0010] S4. Map the relationship and status between devices into a graph structure, and build a collaborative graph model between devices based on the mapping results, introducing the device collaboration regularization term. Constrain the collaborative workload between devices, design an adaptive resource allocation mechanism to dynamically adjust the resource allocation between devices and output a resource allocation strategy * ;

[0011] S5. Build a device anomaly detection model based on resource allocation strategy r * The device status is used to identify potential abnormal activities or operations of the device, and a privacy protection regularization term is introduced. Limit the spread of sensitive data between devices, and then build an adaptive anomaly detection and privacy protection optimization model to dynamically adjust anomaly detection and privacy protection, and obtain the optimized resource allocation strategy r after adding anomaly detection and privacy protection. opt ;

[0012] S6. Construct a reinforcement learning model for resource allocation strategy opt Dynamically adjust the model parameters, and use the device status, resource allocation strategy and user feedback as input to train the reinforcement learning model. Train the policy network π(a|s) of the reinforcement learning model to enable the reinforcement learning model to select the optimal action a under different states s, and then introduce the adaptive regularization term Specially handles the high-dimensional interactivity and dynamics of multi-device states through adaptive regularization terms The weight sharing and dynamic adjustment of the gradient of the reinforcement learning model are systematically adapted to different device states and resource allocations. Then, an adaptive learning optimization strategy is designed to dynamically adjust the model parameters of the reinforcement learning model to obtain the optimized adaptive strategy θ opt , and adjust the adaptive strategy θ based on the feedback of real-time interaction opt Optimize and dynamically adjust to environmental changes and user needs.

[0013] Furthermore, the preprocessing includes standardization and normalization processing, time alignment and interpolation synchronization processing, and feature extraction and dimensionality reduction processing; the multimodal feature vectors are fused into a comprehensive feature vector using a weighted splicing fusion algorithm for data from different sensors, which is expressed as follows:

[0014]

[0015] Among them, F combined represents the integrated feature vector after fusion, N represents the number of sensors, and w i Represents the weight coefficient of the i-th sensor, satisfying F i ′ represents the data after dimensionality reduction.

[0016] Furthermore, the modality relationship mapping network is expressed as follows:

[0017] g k =W k f k +b k

[0018] Among them, f k represents the kth modal feature, g k represents the linearly transformed eigenvector of the kth mode; W k Represents the weight matrix of the linear transformation, with size where d k is the original dimension of the kth modal feature, d′ represents the unified feature dimension; b k is the bias vector of the linear transformation, with size R d ′;

[0019] The feature regularization term It is expressed as follows:

[0020]

[0021] in, represents the feature balance regularization term, γ represents the regularization coefficient, and controls the strength of the regularization term. The square of the linearly transformed eigenvector L of the kth mode 2 norm, which indicates the energy of the modality in the fusion feature, K indicates the number of devices in the current environment, j indicates the jth modality, and g j Represents the linearly transformed eigenvector of the jth mode;

[0022] The adaptive fusion network is constructed as follows:

[0023] The fused feature vector is expressed as follows:

[0024]

[0025] Among them, h fused represents the fused feature vector, which is used for user intention recognition, β k represents the adaptation weight of the k-th mode, and the eigenvector g after the linear transformation of the k-th mode k It is calculated by the inner product of the weight vector u, which is expressed as follows:

[0026]

[0027] Among them, u represents a trainable weight vector with a dimension of Rd′;

[0028] The deep self-attention mechanism includes:

[0029] First, calculate the attention weight matrix A, which is expressed as follows:

[0030]

[0031] Where G = [g 1 , g 2 , ..., g K ] T represents the set matrix of all modal features, GT represents the set matrix transpose, g K Represents the collection matrix of the kth modal features, with a size of A represents the attention weight matrix, with a size of R K×K ;

[0032] Secondly, using the attention weight matrix A, calculate the deep fusion feature h deep , which is expressed as follows:

[0033] h deep =AG

[0034] Finally, the fully connected layer is used to identify the user intent using the deep fusion features to obtain the output y, which is expressed as follows:

[0035] y=σ(W y h deep +b y )

[0036] Among them, y represents the output vector of user intention; W y Represents the weight matrix of the fully connected layer, with size C represents the number of user intent categories; b y Represents the bias vector of the fully connected layer, with a size of R C ; σ(·) represents the softmax activation function, which is used to convert the output into a probability distribution.

[0037] Furthermore, the user behavior causal reasoning model is constructed as follows:

[0038] Define a causal influence matrix C to represent the potential impact of user intention on behavior. The elements of this matrix C ij Calculated by the following formula:

[0039]

[0040] Among them, C ij Indicates user intention y i Next, user behavior B j The causal influence ofj |y i ) represents a given user intention y i Time behavior B j The conditional probability, P(B j ) indicates behavior B j The prior probability of

[0041] The diversity regularization term It is expressed as follows:

[0042]

[0043] in, represents the behavioral diversity regularization term, η represents the regularization coefficient, and controls the weight of diversity in the overall loss function. j represents the jth element in the behavior prediction vector, indicating behavior B j The predicted probability of

[0044] The optimization objective of the deep learning-based adaptive feedback generation network is expressed as follows:

[0045]

[0046] Among them, f * Indicates that the network generates personalized feedback, T represents transposition; Q(s, f) represents the Q value under a given state s and feedback strategy f, which represents the expected return of the system adopting feedback strategy f in the current state; λ represents the trade-off coefficient between the feedback strategy and the behavior prediction; μ represents the regularization term for controlling the diversity of behavior The coefficient of .

[0047] Furthermore, the device collaboration regularization term It is expressed as follows:

[0048]

[0049] in, represents the device collaboration regularization term, which encourages balanced collaborative operations; θ represents the regularization coefficient, which controls the importance of collaboration balance between devices; |V| represents the total number of devices, S i j represents the collaborative capability between device i and device j in the current state;

[0050] The adaptive resource allocation mechanism is expressed as follows:

[0051]

[0052] Among them, r represents the resource allocation vector, r * represents the optimized resource allocation vector, E represents the connection relationship between devices, and w ijrepresents the weight of the edge between devices, represents the collaborative operation cost between devices i and j, and α represents the regularization term for balancing device collaboration The coefficient of β represents the coefficient of balancing resource allocation changes to prevent large fluctuations in resource allocation, r 0 Represents the initial resource allocation vector, which indicates the initial resource allocation state of the system.

[0053] Furthermore, the device anomaly detection model is based on the device anomaly score function A i Determine whether there is an abnormality, the abnormality score function A i , which is expressed as follows:

[0054]

[0055] Among them, A i represents the anomaly score of device i, indicating the degree of anomaly of the device under the current resource allocation; Z represents the normalization coefficient, ensuring that the sum of the anomaly scores is 1; represents the resource allocation value of device j under the optimized resource allocation strategy; s i represents the state vector of device i; s j represents the state vector of device j; ||s i -s j || 2 represents the square of the Euclidean distance between the state vectors of device i and device j; σ represents the parameter that controls the width of the Gaussian kernel function and adjusts the sensitivity of similarity;

[0056] The privacy-preserving regularization term It is expressed as follows:

[0057]

[0058] in, represents the privacy protection regularization term, which limits the spread of sensitive data; ρ represents the regularization coefficient, which controls the weight of privacy protection in the overall optimization objective; represents the amount of data transmitted between device i and device j; I(s i ∈D sens ,s j ∈D sens ) represents the indicator function. When the device state s i and j Included in sensitive data set D sens When it is in, the value is 1, otherwise it is 0; D sens Represents a collection containing all sensitive data;

[0059] The objective function of the adaptive anomaly detection and privacy protection optimization model is expressed as follows:

[0060]

[0061] Among them, r opt Represents the optimized resource allocation strategy; A i represents the anomaly score of device i; λ represents the privacy protection regularization term The coefficient of represents the privacy protection regularization term; K represents the coefficient for balancing resource allocation changes to prevent drastic changes in resource allocation strategies; r represents the candidate solution for the resource allocation vector; r * represents the optimized resource allocation vector; Represents the current resource allocation vector r and the optimized resource allocation vector r * L 2 The square of the norm represents the change in resource allocation.

[0062] Furthermore, the adaptive regularization term It is expressed as follows:

[0063]

[0064] in, represents the adaptive regularization term; ξ represents the regularization coefficient, which controls the weight of adaptive regularization in the overall optimization objective; K represents the number of devices in the current environment; represents the state-action value function Q(s) with respect to the model parameter θ k , a k ), indicating that device k is in state s k The sensitivity of the strategy to the parameters under

[0065] The objective function of the adaptive learning optimization strategy is expressed as follows:

[0066]

[0067] Among them, θ opt represents the optimized model parameters; (s, a) represents the set of states and actions, (s′, a′) represents the set of states and actions at the next moment; r represents the immediate reward, combined with the current strategy r opt The privacy protection and anomaly detection effects under γ are shown in Figure 2; γ represents the discount factor, which adjusts the weight of future returns; λ represents the adaptive regularization term. The coefficient of represents the regularization term used to prevent excessive parameter updates and ensure the stability of parameter optimization; θ 0 represents the initial model parameters; θ represents the model parameters.

[0068] In a second aspect of the present invention, a control device is provided, comprising:

[0069] The data acquisition module is used to collect the interaction data of electronic components from multiple sensors, and pre-process the data to obtain the pre-processed data of multiple sensors. The weighted splicing fusion algorithm is used to fuse the multi-modal feature vectors into a comprehensive feature vector for output;

[0070] The user intention acquisition module is used to construct a modal relationship mapping network to capture the nonlinear relationship between each modal feature in the comprehensive feature vector and introduce a feature regularization term. By constraining the ratio of each modality feature, an adaptive fusion network is used to combine the features of different modalities and a deep self-attention mechanism is used to capture the relationship between modalities, outputting the user intention vector y;

[0071] The user intention inference module is used to build a user behavior causal inference model, input the user intention vector y into the user behavior causal inference model to infer the user's potential behavior, and generate personalized feedback f based on the user's potential behavior using an adaptive feedback generation network based on deep learning * The user behavior causal inference model uses the risk ratio to calculate the change in the probability of behavior under different intentions, and introduces a behavior diversity regularization term into the user behavior causal inference model. constraining the predicted diversity of potential behaviors;

[0072] The resource allocation module is used to map the relationship and status between devices into a graph structure, and build a collaborative graph model between devices based on the mapping results, introducing a device collaboration regularization term. Constrain the collaborative workload between devices, design an adaptive resource allocation mechanism to dynamically adjust the resource allocation between devices and output a resource allocation strategy * ;

[0073] Resource allocation optimization module, used to build equipment anomaly detection model according to resource allocation strategy * The device status is used to identify potential abnormal activities or operations of the device, and a privacy protection regularization term is introduced. Limit the spread of sensitive data between devices, and then build an adaptive anomaly detection and privacy protection optimization model to dynamically adjust anomaly detection and privacy protection, and obtain the optimized resource allocation strategy r after adding anomaly detection and privacy protection. opt ;

[0074] System optimization module, used to build reinforcement learning model for resource allocation strategy optDynamically adjust the model parameters, and use the device status, resource allocation strategy and user feedback as input to train the reinforcement learning model. Train the policy network π(a|s) of the reinforcement learning model to enable the reinforcement learning model to select the optimal action a under different states s, and then introduce the adaptive regularization term Specially handles the high-dimensional interactivity and dynamics of multi-device states through adaptive regularization terms The weight sharing and dynamic adjustment of the gradient of the reinforcement learning model are systematically adapted to different device states and resource allocations. Then, an adaptive learning optimization strategy is designed to dynamically adjust the model parameters of the reinforcement learning model to obtain the optimized adaptive strategy θ opt , and adjust the adaptive strategy θ based on the feedback of real-time interaction opt Optimize and dynamically adjust to environmental changes and user needs.

[0075] In a third aspect of the present invention, there is provided an electronic device, comprising:

[0076] A memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, any one of the control methods described above is executed.

[0077] In a fourth aspect of the present invention, a storage medium is provided. The computer program stored in the storage medium can be executed by one or more processors and can be used to implement any of the control methods described above.

[0078] The beneficial technical effects of the present invention are at least as follows:

[0079] (1) The present invention uses a combination of multimodal sensor fusion and data processing modules, few-sample learning and generative adversarial networks, and can accurately understand the user's multimodal input (voice, gesture, touch, etc.) with very little sample data and generate multiple possible response strategies. This method effectively overcomes the problem of insufficient response capability of existing technologies in the case of data scarcity.

[0080] (2) The present invention uses causal reasoning technology to predict the user's potential behavior through an adaptive user behavior prediction and feedback optimization module, and processes the user's natural language instructions through a deep learning model. This enables the system to generate more personalized and accurate feedback based on understanding complex language structures and contexts, effectively solving the problem of insufficient understanding of user needs in existing systems. In addition, through the device collaborative optimization and self-learning hardware control module, the present invention uses a graph neural network to achieve collaborative optimization among multiple devices, and introduces an attention mechanism to optimize the hardware resource allocation of the device, thereby improving the efficiency and response speed of the system in multi-device collaboration scenarios, and overcoming the limitations of existing multi-device collaborative control methods.

[0081] (3) In the privacy protection and distributed learning modules, the present invention ensures that the user's personal data is not leaked when the model parameters are shared among multiple devices through federated learning technology, thereby protecting the user's privacy and solving the shortcomings of the prior art in data privacy protection.

[0082] (4) The efficient data processing and anomaly detection module of the present invention accelerates the processing and analysis of large-scale user interaction data through quantum machine learning technology, and integrates an anomaly detection mechanism to quickly identify and respond to abnormal behaviors through self-supervised learning and causal reasoning, significantly improving the security of the system and data processing efficiency.

[0083] (5) Through these innovations, the present invention provides a new, intelligent method for interactive control of electronic devices, which effectively overcomes many defects of the prior art and meets the needs of modern users for intelligence, personalization, high efficiency and data privacy protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0085] Figure 1 The present invention is a flow chart of an electronic component interactive control method.

[0086] Figure 2 This is a framework diagram of an interactive control device of the present invention.

[0087] Figure 3 The present invention is a framework diagram of an electronic device. DETAILED DESCRIPTION

[0088] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0089] In one or more embodiments, Figure 1 As shown, a method for interactive control of electronic components is disclosed, the method comprising the following steps S1-S6:

[0090] S1. Collect interaction data of electronic components from multiple sensors and preprocess the data to obtain data after preprocessing of multiple sensors. Use a weighted splicing fusion algorithm for data from different sensors to fuse multimodal feature vectors into a comprehensive feature vector and output it.

[0091] First, data is collected from multiple sensors, including voice sensors, gesture sensors, touch sensors, etc. Since the data ranges and units of different sensors are different, the raw data needs to be standardized. Let the data vector collected from the i-th sensor be x i , using the mean μ i and standard deviation σ i Normalize the data:

[0092]

[0093] Among them, z i is the standardized data vector, μ i is x i The mean value, σ i is x i This process makes the data of all sensors have the same scale, so that the training effect of the model will not be affected by the dimension difference during fusion.

[0094] Furthermore, a common problem with multimodal data is that the data timestamps between different sensors are inconsistent. In order to ensure the accuracy of data fusion, the present invention needs to align the data of all sensors to a unified time axis. Assume that the data point of each sensor is z i (t), where t is a time point on a unified time axis. By linear interpolation, the present invention adjusts the data points so that they are on the same time axis:

[0095]

[0096] Among them, z i (t 0 ) and z i (t 1 ) are two adjacent time points t 0 and t 1 The data on t is the time point on the unified time axis. Through this interpolation method, all sensor data can be aligned to the same time point, making subsequent processing more accurate.

[0097] After alignment and synchronization, the present invention extracts the most representative features for each type of sensor data. For example, for voice data, frequency domain feature extraction methods such as MFCC (Mel-Frequency Cepstral Coefficient) are used; for gesture data, kinematic features such as acceleration and angular velocity are extracted; for touch data, features such as the position and pressure of the touch point are extracted. The feature matrix F is obtained. i After that, the present invention applies dimensionality reduction techniques such as principal component analysis (PCA) to reduce data redundancy and reduce the impact of noise. The feature matrix after dimensionality reduction is denoted as F i ′.

[0098] For different sensors, the present invention uses a weighted concatenation fusion (WCF) algorithm to fuse multimodal feature vectors into a comprehensive feature vector. Assume that the present invention has obtained a reduced-dimensional feature vector F from each sensor. i ′, its weighted fusion is expressed as:

[0099]

[0100] Among them, F combinsd is the integrated feature vector after fusion, N is the number of sensors, and w i is the weight coefficient of the i-th sensor, satisfying These weight coefficients are dynamically calculated based on the mutual information of the features to ensure that the fused feature vector has the maximum amount of information and discrimination.

[0101] Finally, the comprehensive feature vector F after standardization, alignment, feature extraction, dimensionality reduction and fusion is output combined This feature vector has high information content and low redundancy, and serves as the input for the next step of user intent recognition and deep fusion of multimodal data.

[0102] S2. Construct a modal relationship mapping network to capture the nonlinear relationship between each modal feature in the comprehensive feature vector and introduce a feature regularization term By constraining the ratio of each modal feature, an adaptive fusion network is used to combine the features of different modalities and a deep self-attention mechanism is used to capture the relationship between the modalities to output the user intention vector y.

[0103] Specifically, the present invention uses the comprehensive feature vector F output in step 1 combined , the vector has been weighted to fuse the data from multiple sensors to form a feature set containing multimodal information. In order to further utilize these fused features, the present invention constructs a Modal Relationship Mapping Network (MRMN). The core of the network is to capture the nonlinear relationship between the features of each modality.

[0104] For the fused feature vector F combined =[f 1 , f 2 , ..., f K ], where f k Represents the kth modal feature. The present invention first performs a linear transformation on each modal feature:

[0105] g k =W k f k +bk (4)

[0106] Among them, g k is the linearly transformed eigenvector of the kth mode. k is the weight matrix of the linear transformation, with size where d k is the original dimension of the kth modal feature, and d′ is the unified feature dimension. k is the bias vector of the linear transformation, with size Rd′.

[0107] Through this linear transformation, the present invention can ensure that each modal feature is converted into a same dimensional space to facilitate subsequent modal relationship modeling.

[0108] Furthermore, considering that in the process of multimodal data fusion, there may be a problem that a specific modality contributes too much to the final intent recognition. In order to avoid the model from being overly dependent on a certain modality feature, the present invention introduces a new feature regularization term: This regularization term balances the modal features by constraining the proportion of each modal feature in the final fusion feature:

[0109]

[0110] in, is the feature balance regularization term. γ is the regularization coefficient, which controls the strength of the regularization term. is the square of the kth modal feature L 2 norm, indicating the energy of the modality in the fusion feature. K: the number of devices in the current environment. This regularization term aims to balance the contribution of each modality in the fusion feature and avoid the model's over-reliance on certain modality features.

[0111] Furthermore, after completing the linear transformation and regularization constraints of the modal features, an adaptive fusion network (AFN) is used to combine the features of different modalities. The fused feature vector h fused The calculation formula is as follows:

[0112]

[0113] Where: h fused is the fused feature vector, which is ultimately used for user intent recognition. k is the adaptive weight of the kth mode, defined as:

[0114]

[0115] Among them, u is a trainable weight vector with dimension R d ′.

[0116] Among them, the adaptive weight β k Through the modal eigenvector g k It is calculated by the inner product of and weight vector u. This design enables the model to dynamically assign weights to each modality.

[0117] Furthermore, in order to further improve the effectiveness of the fusion features, the Deep Self-Attention Mechanis m , DSAM) to capture the complex relationship between modalities. First, calculate the attention weight matrix A:

[0118]

[0119] Where: G = [g 1 , g 2 , ..., g K ] T is the collection matrix of all modal features, with size R K×d′ . A is the attention weight matrix, size R K×K . Using the attention weight matrix A, calculate the deep fusion feature h deep :

[0120] h deep =AG (9)

[0121] Finally, the fully connected layer is used to identify the user intent using the deep fusion features to obtain the output y:

[0122] y=σ(W y h deep +b y ) (10)

[0123] Where: y is the output vector of user intention. y is the weight matrix of the fully connected layer, with a size of R C×d′ , C is the number of user intent categories. b y is the bias vector of the fully connected layer, with size R C σ(·) is the softmax activation function, which is used to convert the output into a probability distribution.

[0124] Finally, the output user intention vector y is used for behavior prediction and personalized feedback generation in subsequent steps. These outputs provide the possibility distribution of the user's current operation, ensuring the accuracy and responsiveness of the entire system.

[0125] S3. Build a user behavior causal inference model, input the user intention vector y into the user behavior causal inference model to infer the user's potential behavior, and use the deep learning-based adaptive feedback generation network to generate personalized feedback f according to the user's potential behavior. * The user behavior causal inference model uses the risk ratio to calculate the change in the probability of behavior under different intentions, and introduces a behavior diversity regularization term into the user behavior causal inference model. Constraints are placed on the predicted diversity of potential behaviors.

[0126] In order to achieve accurate user behavior prediction, a user behavior causal inference model (Causal Behavior Inference Model, CBIM) is first constructed. This model uses the user intention vector y output in step 2 to infer the user's potential behavior through a causal reasoning framework. The goal of this invention is to establish a causal network to capture the causal relationship between different user intentions and possible behaviors.

[0127] Specifically, a causal influence matrix C is defined to describe the potential impact of user intention on behavior. The elements of this matrix C i j is calculated by the following formula:

[0128]

[0129] Among them, C ij Indicates user intention y i Next, user behavior B j The causal influence of P(B j |y i ) is a given user intention y i Time behavior B j The conditional probability of P(B j ) is behavior B j The prior probability of .

[0130] In order to describe the causal relationship more accurately, the risk ratio form is used here, which can more sensitively capture the changes in the probability of behavior under different intentions.

[0131] Furthermore, using the causal influence matrix C, the present invention calculates the probability of future behavior based on the user's current intention vector y. The behavior prediction vector p is defined as follows:

[0132] p=σ(Cy)(12)

[0133] Where p is the behavior prediction vector, each element represents the probability of a specific behavior. C is the causal influence matrix, with a size of R m×n, where m is the number of behavior categories and n is the number of user intent categories. y is the user intent vector with a size of R n σ(·) is an extended Softmax function with a temperature coefficient, which is used to smooth the prediction distribution and is defined as:

[0134]

[0135] Where T is the temperature coefficient, which adjusts the smoothness of the prediction distribution to address the problem that the behavior distribution may be too sharp.

[0136] Furthermore, in order to not be limited to a single behavior or feedback mode when generating personalized feedback, especially in a multimodal environment, users may exhibit diverse behavior patterns. Therefore, the present invention introduces a behavior diversity regularization term Encourage the system to maintain a certain diversity in behavior prediction. The definition of this regularization term is as follows:

[0137]

[0138] in, is the behavioral diversity regularization term. η is the regularization coefficient, which controls the weight of diversity in the overall loss function. j is the jth element in the behavior prediction vector, indicating behavior B j This regularization term encourages a balanced distribution of behavior probabilities, prevents the model from being too focused on a certain behavior, and maintains the diversity and flexibility of the model output.

[0139] Furthermore, after obtaining the behavior prediction vector p, the present invention generates personalized feedback through an adaptive feedback generation network (AFGN). The network combines the deep reinforcement learning framework to optimize the feedback strategy to better respond to the user's intention and behavior prediction. The optimization goal of the feedback strategy is:

[0140]

[0141] Among them, f * is the optimized feedback strategy vector. Q(s, f) is the Q value given the state s and feedback strategy f, which indicates the expected return of the system adopting feedback strategy f in the current state. λ is the trade-off coefficient between the feedback strategy and the behavior prediction. μ is the regularization term for controlling the diversity of behavior. This optimization goal ensures that the feedback strategy is not only consistent with the predicted user behavior, but also takes into account the diversity of user behavior, avoiding the generated feedback being too single or stereotyped.

[0142] Finally, output the optimized personalized feedback strategy f* , the feedback strategy can guide the device on how to better respond to user needs, such as adjusting device settings, changing the user interface, etc. These feedbacks will serve as input for the next step (device collaborative optimization and resource management) to further optimize the collaborative operation and resource allocation between devices.

[0143] S4. Map the relationship and status between devices into a graph structure, and build a collaborative graph model between devices based on the mapping results, introducing the device collaboration regularization term. Constrain the collaborative workload between devices, design an adaptive resource allocation mechanism to dynamically adjust the resource allocation between devices and output a resource allocation strategy * .

[0144] In a multi-device environment, in order to better coordinate operations, the relationship and status between devices need to be mapped into a graph structure. Define the device coordination graph model (Device Coordination Graph, DCG), which is represented by G = (V, E), where V represents the set of device nodes and E represents the connection relationship between devices (such as data transmission, control signals, etc.).

[0145] For each device node v i ∈V, set its initial state vector s i , the state vector s i Including information such as the current load of the device, connection quality, available resources, etc. The edge weight w between devices i j represents the collaborative operation cost between device i and device j, such as data transmission delay or energy consumption.

[0146] Furthermore, a device state transfer matrix S is introduced to describe the collaboration capability of devices in different states, which is defined as follows:

[0147]

[0148] Among them, S ij It is the cooperation capability of device i and device j in the current state. i and j is the state vector of device i and device j. A is the state transfer matrix, which controls the influence between state vectors.

[0149] For example, if the state vector of device i is s i =[0.8, 0.5], the state vector of device j is s j =[0.4, 0.9], state transfer matrix The inter-device collaboration capability is

[0150] Furthermore, considering the complexity of the collaborative operation between devices, the present invention introduces a device collaboration regularization term Used to balance the collaborative workload between devices. This regularization term is intended to prevent some devices from being overloaded or underloaded during the collaborative process. Defined as:

[0151]

[0152] in, is the device collaboration regularization term, which encourages balanced collaborative operations. θ is the regularization coefficient, which controls the importance of collaborative balance between devices. |V| is the total number of devices. This regularization term ensures that each device has a reasonable burden in a multi-device system, preventing a single device from becoming a bottleneck.

[0153] Furthermore, based on the collaborative graph model and feedback strategy between devices * , the present invention designs an adaptive resource allocation mechanism (Adaptive Resource Allocation Mechanism, ARAM). This mechanism is used to dynamically adjust the resource allocation between devices to maximize the overall system performance. The resource allocation optimization objective function is defined as:

[0154]

[0155] Among them, r * is the optimized resource allocation vector. ij is the weight of the edge between devices, representing the cost of collaborative operation between devices i and j. α is the regularization term for balancing device collaboration β is the coefficient for balancing resource allocation changes and preventing large fluctuations in resource allocation. r0 is the initial resource allocation vector, which represents the initial resource allocation state of the system. The optimization objective function aims to minimize the total cost of collaborative operation between devices while considering the balance of device collaboration and the stability of resource allocation.

[0156] Through the above resource allocation optimization mechanism, the optimized resource allocation strategy r is output * ,This policy guides each device on how to allocate computing, storage, and communication resources to ensure the efficient operation of the system in a multi-device environment.,The output policy will be used in the next step (real-time anomaly detection and privacy protection) to further ensure the security of the system and the,privacy of user data.

[0157] S5. Build a device anomaly detection model based on resource allocation strategy r * The device status is used to identify potential abnormal activities or operations of the device, and a privacy protection regularization term is introduced. Limit the spread of sensitive data between devices, and then build an adaptive anomaly detection and privacy protection optimization model to dynamically adjust anomaly detection and privacy protection, and obtain the optimized resource allocation strategy r after adding anomaly detection and privacy protection. opt .

[0158] Specifically, use the output resource allocation strategy r from step 4 * , the present invention establishes a device anomaly detection model. This model uses the relationship between device status and resource allocation strategy to identify potential abnormal activities or operations. Set the status of each device to s i , the resource allocation vector is r * , define the abnormal score function A of the device i as follows:

[0159]

[0160] in, The anomaly score of device i indicates the degree of anomaly of the device under the current resource allocation. Z: Normalization coefficient, ensuring that the sum of anomaly scores is 1. Resource allocation value of device j under the optimized resource allocation strategy. i : The state vector of device i, including information such as device load, connection quality, and available resources. j : The state vector of device j. ||s i -s j || 2 : The square of the Euclidean distance between the state vectors of device i and device j. σ: Parameter that controls the width of the Gaussian kernel function and adjusts the sensitivity of similarity.

[0161] For example, if the device state vector s 1 =[0.7, 0.4], s 2 =[0.5, 0.6], resource allocation strategy r * =[0.3, 0.7], and σ = 0.1, then the anomaly score of device 1 is:

[0162]

[0163] Furthermore, in order to protect the privacy of user data in a multi-device environment, the present invention introduces a privacy protection regularization term This regularization term is used to limit the spread of sensitive data between devices and ensure the protection of privacy information during data transmission. Privacy protection regularization term The definition is as follows:

[0164]

[0165] in, Privacy protection regularization term, which limits the spread of sensitive data. ρ: Regularization coefficient, which controls the weight of privacy protection in the overall optimization objective. The amount of data transferred between device i and device j. Indicator function, when the device status s i and j Included in sensitive data collection If it is, the value is 1, otherwise it is 0. A set containing all sensitive data. This regularization term ensures that the system does not leak sensitive information when transmitting data between devices.

[0166] Furthermore, considering the requirements of device anomaly detection and privacy protection, the present invention designs an adaptive anomaly detection and privacy protection optimization framework (Adaptive Anomaly Detection and Privacy Protection Framework, AADPP). The framework achieves dynamic adjustment of anomaly detection and privacy protection by optimizing the objective function.

[0167] The optimization objective function is defined as:

[0168]

[0169] Among them, r opt : Optimized resource allocation strategy. The anomaly score of device i. λ: Privacy-preserving regularization term The coefficient of . Privacy protection regularization term. κ: Coefficient for balancing resource allocation changes to prevent drastic changes in resource allocation strategies. r: Candidate solutions for resource allocation vectors. * : The optimized resource allocation strategy output by the fourth step. Current resource allocation vector r and optimized resource allocation vector r * L 2 The square of the norm represents the change in resource allocation.

[0170] Finally, output the optimized resource allocation strategy r opt , which is used to guide the system to perform real-time anomaly detection and privacy protection operations. This strategy can effectively detect abnormal device behavior and prevent sensitive data from being leaked in a multi-device environment.

[0171] S6. Construct a reinforcement learning model for resource allocation strategy opt Dynamically adjust the model parameters, and use the device status, resource allocation strategy and user feedback as input to train the reinforcement learning model. Train the policy network π(a|s) of the reinforcement learning model to enable the reinforcement learning model to select the optimal action a under different states s, and then introduce the adaptive regularization term Specially handles the high-dimensional interactivity and dynamics of multi-device states through adaptive regularization terms The weight sharing and dynamic adjustment of the gradient of the reinforcement learning model are systematically adapted to different device states and resource allocations. Then, an adaptive learning optimization strategy is designed to dynamically adjust the model parameters of the reinforcement learning model to obtain the optimized adaptive strategy θ opt , and adjust the adaptive strategy θ based on the feedback of real-time interaction opt Optimize and dynamically adjust to environmental changes and user needs.

[0172] Specifically, using the resource allocation strategy r output in the fifth step opt The present invention constructs a reinforcement learning model (RLM) to dynamically adjust system parameters to optimize overall performance. The model takes information such as device status, resource allocation strategy, and user feedback as input to train a policy network π(a|s) to select the optimal action a under different states s.

[0173] First, we define the state-action value function Q(s, a) to evaluate the expected reward of performing action a in state s. To capture the complexity in a multi-device environment, state s is represented by the device state vector s i , optimization strategy opt As well as privacy protection and anomaly detection feedback. The state-action value function adopts the following update rule:

[0174]

[0175] Where Q(s, a) is the value function of executing action a in state s. α is the learning rate, which controls the update step size. r is the immediate reward, which is combined with the optimization strategy r. opt The privacy protection effect and anomaly detection accuracy under γ are calculated. γ: discount factor, weighing the impact of future rewards. s′: the new state reached after executing action a. a′: the optimal action under the new state s′. The purpose of the reinforcement learning framework is to adjust the strategy in real time according to the current system state in a multi-device collaborative environment to maximize the overall performance.

[0176] Furthermore, in order to improve the adaptability of the model to multi-device collaborative environments, an innovative adaptive regularization term is introduced Specially designed to handle the high-dimensional interactivity and dynamics of multiple device states. This regularization term enhances the system's adaptability to different device states and resource allocations through weight sharing and dynamic adjustment mechanisms.

[0177] Adaptive Regularization The definition is as follows:

[0178]

[0179] in, Adaptive regularization term, which enhances the generalization ability of the model to different device states. ξ: Regularization coefficient, which controls the weight of adaptive regularization in the overall optimization objective. K: The number of devices in the current environment. The state-action value function Q(s) about the model parameter θ k , a k ), indicating that device k is in state s k The sensitivity of the strategy to the parameters.

[0180] For example, if the number of devices K = 3, the state-action value function gradients are Then the adaptive regularization term is:

[0181]

[0182] This regularization term is designed to encourage the model to maintain a balanced strategy across different device states and prevent over-reliance on a specific device state.

[0183] Furthermore, based on the state-action value function Q(s, a) and the adaptive regularization term The present invention designs an adaptive learning optimization strategy (ALOS) to dynamically adjust model parameters during system operation. The optimization objective function is defined as:

[0184]

[0185] Among them, θ opt : Optimized model parameters. (s, a): A collection of states and actions. r: Instant reward, combined with the current strategy r opt Privacy protection and anomaly detection effects under γ: discount factor, adjusting the weight of future rewards. λ: adaptive regularization term The coefficient of . Regularization term used to prevent excessive parameter updates and ensure the stability of parameter optimization. 0 : Initial model parameters. This optimization objective function takes into account the immediate return, adaptive regularization term and parameter stability, ensuring that the system takes into account learning efficiency, generalization ability and stability during the adaptive learning process.

[0186] Furthermore, after obtaining the optimized adaptive strategy θ opt After that, the system will continue to monitor the device status, resource allocation and user behavior in real time, and dynamically adjust the strategy using the reinforcement learning model. The system will continuously optimize the strategy based on real-time feedback.opt , in order to cope with environmental changes and dynamic adjustments of user needs, ensuring the long-term stability and optimal performance of the system.

[0187] The final output optimization strategy θ opt It can effectively perform adaptive learning and self-optimization in a multi-device environment, improving the overall response speed and resource utilization of the system.

[0188] Based on the foregoing embodiments, an embodiment of the present application provides a control device, and the modules included in the device and the units included in the modules can be implemented by a processor in a computer device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0189] The present application embodiment provides a control device, Figure 2 A schematic diagram of the structure of a control device provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the control device includes:

[0190] The data acquisition module 101 is used to collect the interaction data of the electronic components from multiple sensors, and pre-process the data to obtain the pre-processed data of multiple sensors, and use the weighted splicing fusion algorithm to fuse the multi-modal feature vectors into a comprehensive feature vector for output;

[0191] The user intention acquisition module 102 is used to construct a modal relationship mapping network to capture the nonlinear relationship between each modal feature in the comprehensive feature vector and introduce a feature regularization term. By constraining the ratio of each modality feature, an adaptive fusion network is used to combine the features of different modalities and a deep self-attention mechanism is used to capture the relationship between modalities, outputting the user intention vector y;

[0192] The user intention inference module 103 is used to construct a user behavior causal inference model, input the user intention vector y into the user behavior causal inference model to infer the user's potential behavior, and generate personalized feedback f based on the user's potential behavior using an adaptive feedback generation network based on deep learning. * The user behavior causal inference model uses the risk ratio to calculate the change in the probability of behavior under different intentions, and introduces a behavior diversity regularization term into the user behavior causal inference model. constraining the predicted diversity of potential behaviors;

[0193] The resource allocation module 104 is used to map the relationship and status between devices into a graph structure, and build a collaborative graph model between devices based on the mapping results, and introduce a device collaboration regularization term. Constrain the collaborative workload between devices, design an adaptive resource allocation mechanism to dynamically adjust the resource allocation between devices and output a resource allocation strategy * ;

[0194] Resource allocation optimization module 105 is used to build a device anomaly detection model according to the resource allocation strategy r * The device status is used to identify potential abnormal activities or operations of the device, and a privacy protection regularization term is introduced. Limit the spread of sensitive data between devices, and then build an adaptive anomaly detection and privacy protection optimization model to dynamically adjust anomaly detection and privacy protection, and obtain the optimized resource allocation strategy r after adding anomaly detection and privacy protection. opt ;

[0195] System optimization module 106, used to build a reinforcement learning model for resource allocation strategy r opt Dynamically adjust the model parameters, and use the device status, resource allocation strategy and user feedback as input to train the reinforcement learning model. Train the policy network π(a|s) of the reinforcement learning model to enable the reinforcement learning model to select the optimal action a under different states s, and then introduce the adaptive regularization term Specially handles the high-dimensional interactivity and dynamics of multi-device states through adaptive regularization terms The weight sharing and dynamic adjustment of the gradient of the reinforcement learning model are systematically adapted to different device states and resource allocations. Then, an adaptive learning optimization strategy is designed to dynamically adjust the model parameters of the reinforcement learning model to obtain the optimized adaptive strategy θ opt , and adjust the adaptive strategy θ based on the feedback of real-time interaction opt Optimize and dynamically adjust to environmental changes and user needs.

[0196] It should be noted that in the embodiment of the present application, if the above-mentioned control method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, ReadOnlyMemory), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0197] Accordingly, an embodiment of the present application provides a storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps in the control method provided in the above embodiment are implemented.

[0198] An embodiment of the present application provides an electronic device; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the electronic device 300 includes: a processor 301, at least one communication bus 302, a user interface 303, at least one external communication interface 304, and a memory 305. The communication bus 302 is configured to realize the connection and communication between these components. The user interface 303 may include a display screen, and the external communication interface 304 may include a standard wired interface and a wireless interface. The processor 301 is configured to execute the program of the control method stored in the memory to implement the steps in the control method provided in the above embodiment.

[0199] It should be noted here that the description of the above storage medium and electronic device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0200] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0201] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, object or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, object or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, object or device including the element.

[0202] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0203] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0204] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0205] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read Only Memory), disks or optical disks, and other media that can store program codes.

[0206] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a controller to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0207] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An electronic component interactive control method, characterized in that: The method comprises the following steps: S1. Collect interaction data of electronic components from multiple sensors, preprocess the data, obtain preprocessed data of multiple sensors, use weighted splicing fusion algorithm for data from different sensors to fuse multimodal feature vectors into a comprehensive feature vector, and output it; S2. Construct a modal relationship mapping network to capture the nonlinear relationship between each modal feature in the comprehensive feature vector and introduce a feature regularization term By constraining the ratio of each modality feature, an adaptive fusion network is used to combine the features of different modalities and a deep self-attention mechanism is used to capture the relationship between modalities, outputting the user intention vector y; S3. Build a user behavior causal inference model, input the user intention vector y into the user behavior causal inference model to infer the user's potential behavior, and use the deep learning-based adaptive feedback generation network to generate personalized feedback f according to the user's potential behavior. * ; The user behavior causal inference model uses the risk ratio to calculate the change in the probability of behavior under different intentions, and introduces a behavior diversity regularization term into the user behavior causal inference model. constraining the predicted diversity of potential behaviors; S4. Map the relationship and status between devices into a graph structure, and build a collaborative graph model between devices based on the mapping results, introducing the device collaboration regularization term. Constrain the collaborative workload between devices, design an adaptive resource allocation mechanism to dynamically adjust the resource allocation between devices and output a resource allocation strategy * ; S5. Build a device anomaly detection model based on resource allocation strategy r * The device status is used to identify potential abnormal activities or operations of the device, and a privacy protection regularization term is introduced. Limit the spread of sensitive data between devices, and then build an adaptive anomaly detection and privacy protection optimization model to dynamically adjust anomaly detection and privacy protection, and obtain the optimized resource allocation strategy r after adding anomaly detection and privacy protection. opt ; S6. Construct a reinforcement learning model for resource allocation strategy opt Dynamically adjust the model parameters, and use the device status, resource allocation strategy and user feedback as input to train the reinforcement learning model. Train the policy network π(a|s) of the reinforcement learning model to enable the reinforcement learning model to select the optimal action a under different states s, and then introduce the adaptive regularization term Specially handles the high-dimensional interactivity and dynamics of multi-device states through adaptive regularization terms The weight sharing and dynamic adjustment of the gradient of the reinforcement learning model are systematically adapted to different device states and resource allocations. Then, an adaptive learning optimization strategy is designed to dynamically adjust the model parameters of the reinforcement learning model to obtain the optimized adaptive strategy θ opt , and adjust the adaptive strategy θ based on the feedback of real-time interaction opt Optimize and dynamically adjust to environmental changes and user needs.

2. The electronic component interactive control method according to claim 1, characterized in that: The preprocessing includes standardization and normalization processing, time alignment and interpolation synchronization processing, and feature extraction and dimensionality reduction processing; the weighted splicing fusion algorithm is used for the data of different sensors to fuse the multimodal feature vectors into a comprehensive feature vector, which is expressed as follows: Among them, F combined represents the integrated feature vector after fusion, N represents the number of sensors, and w i Represents the weight coefficient of the i-th sensor, satisfying F i ′ represents the data after dimensionality reduction.

3. The electronic component interactive control method according to claim 1, characterized in that: The modal relationship mapping network is expressed as follows: g k =W k f k +b k Among them, f k represents the kth modal feature, g k represents the linearly transformed eigenvector of the kth mode; W k Represents the weight matrix of the linear transformation, with size where d k is the original dimension of the kth modal feature, d′ represents the unified feature dimension; b k is the bias vector of the linear transformation, with size R d′ ; The feature regularization term It is expressed as follows: in, represents the feature balance regularization term, γ represents the regularization coefficient, and controls the strength of the regularization term. represents the square L2 norm of the linearly transformed feature vector of the kth mode, which represents the energy of the mode in the fusion feature. K represents the number of devices in the current environment, j represents the jth mode, and g j Represents the linearly transformed eigenvector of the jth mode; The adaptive fusion network is constructed as follows: The fused feature vector is expressed as follows: Among them, h fused represents the fused feature vector, which is used for user intention recognition, β k represents the adaptive weight of the k-th mode, and the eigenvector g after the linear transformation of the k-th mode k It is calculated by the inner product of the weight vector u, which is expressed as follows: Among them, u represents the trainable weight vector, the dimension is R d′ ; The deep self-attention mechanism includes: First, calculate the attention weight matrix A, which is expressed as follows: Where G = [g1, g2, …, g K ] T represents the set matrix of all modal features, GT represents the set matrix transpose, g K Represents the collection matrix of the kth modal features, with a size of A represents the attention weight matrix, with a size of R K×K ; Secondly, using the attention weight matrix A, calculate the deep fusion feature h deep , which is expressed as follows: h deep =AG Finally, the fully connected layer is used to identify the user intent using the deep fusion features to obtain the output y, which is expressed as follows: y=σ(W y h deep +b y ) Among them, y represents the output vector of user intention; W y Represents the weight matrix of the fully connected layer, with size C represents the number of user intent categories; b y Represents the bias vector of the fully connected layer, with a size of R C ; σ(·) represents the softmax activation function, which is used to convert the output into a probability distribution.

4. The electronic component interactive control method according to claim 1, characterized in that: The user behavior causal reasoning model is constructed as follows: Define a causal influence matrix C to represent the potential impact of user intention on behavior. The elements of this matrix C ij Calculated by the following formula: Among them, C ij Indicates user intention y i Next, user behavior B j The causal influence of j |y i ) represents a given user intention y i Time behavior B j The conditional probability, P(B j ) indicates behavior B j The prior probability of The diversity regularization term It is expressed as follows: in, represents the behavioral diversity regularization term, η represents the regularization coefficient, and controls the weight of diversity in the overall loss function. j represents the jth element in the behavior prediction vector, indicating behavior B j The predicted probability of The optimization objective of the deep learning-based adaptive feedback generation network is expressed as follows: Among them, f * Indicates that the network generates personalized feedback; T represents transposition; Q(s,f) represents the Q value under a given state s and feedback strategy f, which represents the expected return of the system adopting feedback strategy f in the current state; λ represents the trade-off coefficient between the feedback strategy and the behavior prediction; μ represents the regularization term for controlling the diversity of behavior The coefficient of .

5. The electronic component interactive control method according to claim 1, characterized in that: The device cooperative regularization term It is expressed as follows: in, represents the device collaboration regularization term, which encourages balanced collaborative operations; θ represents the regularization coefficient, which controls the importance of collaboration balance between devices; |V| represents the total number of devices, S ij Indicates the collaboration capability between device i and device j in the current state; The adaptive resource allocation mechanism is expressed as follows: Among them, r represents the resource allocation vector, r * represents the optimized resource allocation vector, E represents the connection relationship between devices, and w ij represents the weight of the edge between devices, represents the collaborative operation cost between devices i and j, and α represents the regularization term for balancing device collaboration , β represents the coefficient of balancing resource allocation changes to prevent large fluctuations in resource allocation, and r0 represents the initial resource allocation vector, which represents the initial resource allocation state of the system.

6. The electronic component interactive control method according to claim 5, characterized in that: The device anomaly detection model is based on the device anomaly score function A i Determine whether there is an abnormality, the abnormality score function A i , which is expressed as follows: Among them, A i represents the anomaly score of device i, indicating the degree of anomaly of the device under the current resource allocation; Z represents the normalization coefficient, ensuring that the sum of the anomaly scores is 1; represents the resource allocation value of device j under the optimized resource allocation strategy; s i represents the state vector of device i; s j Represents the state vector of device j; ∥s i -s j ∥ 2 represents the square of the Euclidean distance between the state vectors of device i and device j; σ represents the parameter that controls the width of the Gaussian kernel function and adjusts the sensitivity of similarity; The privacy-preserving regularization term It is expressed as follows: in, represents the privacy protection regularization term, which limits the spread of sensitive data; ρ represents the regularization coefficient, which controls the weight of privacy protection in the overall optimization objective; represents the amount of data transmitted between device i and device j; I(s i ∈D sens ,s j ∈D sens ) represents the indicator function. When the device state s i and j Included in sensitive data set D sens When it is in, the value is 1, otherwise it is 0; D sens Represents a collection containing all sensitive data; The objective function of the adaptive anomaly detection and privacy protection optimization model is expressed as follows: Among them, r opt Represents the optimized resource allocation strategy; A i represents the anomaly score of device i; λ represents the privacy protection regularization term The coefficient of represents the privacy protection regularization term; κ represents the coefficient for balancing resource allocation changes to prevent drastic changes in resource allocation strategies; r represents the candidate solution for the resource allocation vector; r * represents the optimized resource allocation vector; Represents the current resource allocation vector r and the optimized resource allocation vector r * The square of the L2 norm between , represents the change in resource allocation.

7. The electronic component interactive control method according to claim 1, characterized in that: The adaptive regularization term It is expressed as follows: in, represents the adaptive regularization term; ξ represents the regularization coefficient, which controls the weight of adaptive regularization in the overall optimization objective; K represents the number of devices in the current environment; represents the state-action value function Q(s) with respect to the model parameter θ k ,a k ), indicating that device k is in state s k The sensitivity of the strategy to the parameters under The objective function of the adaptive learning optimization strategy is expressed as follows: Among them, θ opt represents the optimized model parameters; (s, a) represents the set of states and actions, (s′, a′) represents the set of states and actions at the next moment; r represents the immediate reward, combined with the current strategy r opt The privacy protection and anomaly detection effects under γ are shown in Figure 2; γ represents the discount factor, which adjusts the weight of future returns; λ represents the adaptive regularization term. The coefficient of represents the regularization term used to prevent excessive parameter updating and ensure the stability of parameter optimization; θ0 represents the initial model parameters; θ represents the model parameters.

8. A control device, characterized in that: include: The data acquisition module is used to collect the interaction data of electronic components from multiple sensors, and pre-process the data to obtain the pre-processed data of multiple sensors. The weighted splicing fusion algorithm is used to fuse the multi-modal feature vectors into a comprehensive feature vector for output; The user intention acquisition module is used to construct a modal relationship mapping network to capture the nonlinear relationship between each modal feature in the comprehensive feature vector and introduce a feature regularization term. By constraining the ratio of each modality feature, an adaptive fusion network is used to combine the features of different modalities and a deep self-attention mechanism is used to capture the relationship between modalities, outputting the user intention vector y; The user intention inference module is used to build a user behavior causal inference model, input the user intention vector y into the user behavior causal inference model to infer the user's potential behavior, and generate personalized feedback f based on the user's potential behavior using an adaptive feedback generation network based on deep learning * ; The user behavior causal inference model uses the risk ratio to calculate the change in the probability of behavior under different intentions, and introduces a behavior diversity regularization term into the user behavior causal inference model. constraining the predicted diversity of potential behaviors; The resource allocation module is used to map the relationship and status between devices into a graph structure, and build a collaborative graph model between devices based on the mapping results, introducing a device collaboration regularization term. Constrain the collaborative workload between devices, design an adaptive resource allocation mechanism to dynamically adjust the resource allocation between devices and output a resource allocation strategy * ; Resource allocation optimization module, used to build equipment anomaly detection model according to resource allocation strategy * The device status is used to identify potential abnormal activities or operations of the device, and a privacy protection regularization term is introduced. Limit the spread of sensitive data between devices, and then build an adaptive anomaly detection and privacy protection optimization model to dynamically adjust anomaly detection and privacy protection, and obtain the optimized resource allocation strategy r after adding anomaly detection and privacy protection. opt ; System optimization module, used to build reinforcement learning model for resource allocation strategy opt Dynamically adjust the model parameters, and use the device status, resource allocation strategy and user feedback as input to train the reinforcement learning model. Train the policy network π(a|s) of the reinforcement learning model to enable the reinforcement learning model to select the optimal action a under different states s, and then introduce the adaptive regularization term Specially handles the high-dimensional interactivity and dynamics of multi-device states through adaptive regularization terms The weight sharing and dynamic adjustment of the gradient of the reinforcement learning model are systematically adapted to different device states and resource allocations. Then, an adaptive learning optimization strategy is designed to dynamically adjust the model parameters of the reinforcement learning model to obtain the optimized adaptive strategy θ opt , and adjust the adaptive strategy θ based on the feedback of real-time interaction opt Optimize and dynamically adjust to environmental changes and user needs.

9. An electronic device, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the control method according to any one of claims 1 to 7 is executed.

10. A storage medium, characterized in that: The computer program stored in the storage medium can be executed by one or more processors and can be used to implement the control method as described in any one of claims 1 to 7.

Images