A method and device for constructing a multi-source domain human posture recognition migration model

Abstract

The present invention relates to the field of data analysis, and in particular to a method and device for constructing a multi-source domain human posture recognition migration model; using a pool transformation formula to generate a pool representation of multiple source domain data and target domain data of human posture; calculating the RSD distance between the pool representations of each source domain and the target domain, selecting the pool representation of the first similar source domain according to the RSD distance and enhancing it using the mix-up formula; selecting the second similar source domain from the pool representations of other source domains except the first similar source domain and enhancing it; merging the enhanced pool representations of the first and second similar source domains as the pool representation of the most similar source domain and inputting them into a TSK fuzzy system to obtain an initial human posture recognition model; adjusting the consequent parameters of the initial fuzzy rules of the model according to the pool representation of the target domain to obtain an optimized human posture recognition model. Rapid and accurate human posture recognition based on multiple insufficient source domains is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a method and device for constructing a multi-source domain human posture recognition migration model. Background Art

[0002] Human posture recognition refers to the process of identifying and understanding human posture by analyzing and processing information from different parts of the human body. This technology is widely used in many fields such as video surveillance, human behavior analysis, virtual reality, motion capture, and medical rehabilitation. Human posture recognition obtains human posture information by processing multi-channel sensory signals, and can accurately track the dynamic changes of the human body, thereby providing support for many application scenarios. In recent years, with the popularization of intelligent devices and applications, the importance of human posture recognition technology in security monitoring, entertainment games, sports training, and health monitoring has become increasingly prominent. For example, in security monitoring, accurate human posture recognition can help identify abnormal behavior and prevent potential safety hazards; in sports training, posture recognition technology can provide athletes with more accurate motion analysis to help them improve training results. Therefore, human posture recognition is of great significance to improving the accuracy and effectiveness of human-computer interaction.

[0003] At present, the commonly used human posture recognition methods include the following two categories: (1) Image-based recognition methods, such as 2D key point detection and 3D posture estimation. These methods extract the key points of the human skeleton in images or videos to infer the human posture. 2D key point detection is usually easily affected by perspective and occlusion, and the recognition accuracy is easily reduced. Although 3D posture estimation can more accurately reflect the actual posture of the human body, its computational complexity is high and real-time performance is difficult to guarantee. (2) Sensor-based recognition methods, such as inertial measurement unit sensors, depth cameras and lidar. These methods collect data such as acceleration, angular velocity, depth information, etc. of the human body in motion to perform posture recognition. Although these sensors can provide relatively stable recognition results in complex environments, the memory capacity of these devices is limited and cannot store a large amount of historical data, resulting in only limited data sets being processed. In the case of insufficient data, the recognition accuracy and adaptability of the model are easily affected. Especially when facing new application scenarios or users, it is often necessary to re-collect and annotate a large amount of data, which increases the complexity and cost of the system.

[0004] In order to solve the above shortcomings, reservoir computing provides an effective solution for human posture recognition methods. As an effective method for concisely designing recurrent neural networks, reservoir computing can achieve fast training without affecting accuracy. The input weight matrix and the reservoir layer weight matrix of the reservoir computing remain unchanged after being initialized, and only the output weight matrix needs to be trained. However, its efficiency depends on a large amount of training data, otherwise it will affect the generalization of the reservoir computing model. In some emerging fields or unstable environments, such as real-time and practical human posture recognition scenarios, we often face data shortages. To address the data shortage problem, transfer learning technology can improve the generalization ability of the model by utilizing existing multiple source domain data, but what is more likely to happen in real life is that the multiple source domains we have may also be insufficient. At the same time, most of the existing transfer learning technologies are based on deep learning networks, but the existing deep learning methods have many parameters, large models, difficult deployment, and long training and reasoning time, so they are not suitable for edge computing environments that require real-time detection.

[0005] In summary, it is difficult for existing human posture recognition methods to achieve fast and accurate human posture recognition based on multiple insufficient source domain data, and this problem needs to be solved urgently. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the prior art is difficult to achieve fast and accurate human posture recognition based on multiple insufficient source domain data.

[0007] In order to solve the above technical problems, the present invention provides a method for constructing a multi-source domain human posture recognition migration model, which specifically includes the following steps:

[0008] Based on multiple source domain data and target domain data of human body posture, the reservoir pool representation of each source domain and the reservoir pool representation of the target domain are generated respectively by using the reservoir pool transformation formula;

[0009] Calculate the RSD distance between the pool representation of each source domain and the pool representation of the target domain, select the pool representation of the source domain with the smallest RSD distance with the pool representation of the target domain as the pool representation of the first similar source domain, and use the mix-up formula to enhance the pool representation of the first similar source domain; use the RSD distance between the enhanced pool representation of the first similar source domain and the pool representation of the target domain as the reference RSD distance, and use the remaining source domains except the first similar source domain as the candidate source domain set;

[0010] Taking the RSD distance between the enhanced pool representation of each source domain in the candidate source domain set and the pool representation of the target domain as less than the reference RSD distance as the goal, respectively obtain the mix-up ratio range of the pool representation of each source domain in the candidate source domain set; taking the pool representation of the source domain in the candidate source domain set whose mix-up ratio range intersects with the preset ratio range as the pool representation of the second similar source domain; based on the mix-up ratio range of the pool representation of the second similar source domain, using the mix-up formula to enhance the pool representation of the second similar source domain;

[0011] The enhanced pool representations of the first and second similar source domains are merged as the pool representation of the most similar source domain; the pool representation of the most similar source domain is input into the TSK fuzzy system to obtain an initial human posture recognition model;

[0012] The consequent parameters of the initial fuzzy rules in the initial human posture recognition model are adjusted according to the reserve pool representation of the target domain to obtain an optimized human posture recognition model.

[0013] Preferably, the calculating the RSD distance between the reservoir pool representation of each source domain and the reservoir pool representation of the target domain comprises:

[0014] ,

[0015] ,

[0016] ,

[0017] ,

[0018] in, and Respectively represent A pool representation of the source domain and a pool representation of the target domain; represents the RSD distance between the reservoir pool representation of the sth source domain and the reservoir pool representation of the target domain; Indicates about The mathematical expectation of is the first sample sampled from the reserve pool representation of the target domain; is a second sample sampled from the reserve pool representation of the target domain; Indicates s Stein scoring function for the pool representation of source domains; is the source domain density; and Respectively represent The number of samples in the source domain and the The source domain A reserve pool is represented; and are the kernel function and its bandwidth respectively; Representation function right The gradient of Representation function right The gradient of Representation function right The gradient of trace It means finding the trace of a matrix; express right The gradient of Indicates transpose.

[0019] Preferably, the mix-up formula is specifically expressed as follows:

[0020] ,

[0021] in, and They are the pool representation sampled from the first and second similar source domains and the one-hot encoding of the corresponding real sample labels; and They are the one-hot encodings of the pool representation sampled from the target domain and the corresponding true sample labels; Reservoir representation representing the enhanced source domain; The sample labels of the reserve pool representation of the enhanced source domain; Represents the mix-up ratio when enhancing the first similar source domain. Random sampling from 0-1; when enhancing the second similar source domain, Randomly sample from the intersection of the mix-up ratio range of the second most similar source domain and 0-1.

[0022] Preferably, the calculation formula of the RSD distance between the enhanced reserve pool representation of each source domain in the selected source domain set and the reserve pool representation of the target domain is:

[0023] ,

[0024] ,

[0025] ,

[0026] in, Indicates the first s The RSD distance between the enhanced reservoir representation of the source domain and the reservoir representation of the target domain; Indicates the firsts The reserve pool representation after the source domain is enhanced; A reservoir representation representing the target domain; Represents the reserve pool representation of the s-th source domain in the candidate source domain set; Indicates s The RSD distance between the source domain’s reservoir representation and the target domain’s reservoir representation; Indicates the first s The reserve pool of source domains represents the mix-up ratio for enhancement; Indicates about The mathematical expectation of is the first sample sampled from the reserve pool representation of the target domain; , The size is The jth column of the identity matrix, m is the dimension represented by the reservoir; Indicates s Stein scoring function for the pool representation of source domains; represents the source domain density; Indicates s The number of samples in the source domain; and are the kernel function and its bandwidth respectively; Indicates s The source domain A reserve pool is represented; The target domain A reserve pool is represented; express right The gradient of represents transpose;

[0027] The RSD distance between the enhanced pool representation of each source domain and the pool representation of the target domain is less than the reference RSD distance, and the target representation is:

[0028] ,

[0029] in, represents the enhanced reserve pool representation of the first similar source domain; Represents the RSD distance between the enhanced reservoir representation of the first similar source domain and the reservoir representation of the target domain.

[0030] Preferably, the preset ratio range is [0, 1].

[0031] Preferably, the initial fuzzy rule is expressed as follows:

[0032] ,

[0033] ,

[0034] ，

[0035] in, Indicates k Under the fuzzy rule m dimensional features, defined by a membership function; is the first sample sampled from the reserve pool representation of the target domain; express The m-th dimension feature of is the cth first-order linear output under the kth fuzzy rule; It is k Under the fuzzy rule c The output j coefficients; C Represents the number of first-order linear outputs of the fuzzy rule.

[0036] Preferably, adjusting the consequent parameters of the initial fuzzy rules in the initial human posture recognition model according to the reserve pool representation of the target domain specifically includes:

[0037] The empirical risk term represented by the reserve pool of the target domain and the knowledge regularization term of the historical consequent parameter migration are used as the objective function to solve the objective function;

[0038] The consequent parameters of the initial fuzzy rules in the initial human posture recognition model are adjusted according to the solution of the objective function.

[0039] Preferably, the objective function expression is as follows:

[0040] ,

[0041] in, It is the reserve pool representation of the target domain after the fuzzification operation; Represents the one-hot encoding of the true labels corresponding to all samples in the target domain; are the consequent parameters of the fuzzy rules trained on the selected most similar source domain’s reservoir representation and its corresponding true label; The consequent parameters of the fuzzy rules represented by the reservoir that are applicable to the target domain; Represents preset hyperparameters.

[0042] Preferably, the solution formula of the objective function is as follows:

[0043] ,

[0044] in, It is the reserve pool representation of the target domain after the fuzzification operation; represents the transpose of the reservoir representation of the target domain after the fuzzification operation; Represents the one-hot encoding of the true labels corresponding to all samples in the target domain; Represents the transpose of the one-hot encoding of the true labels corresponding to all samples in the target domain; Indicates size The identity matrix of are the consequent parameters of the fuzzy rules trained on the selected most similar source domain’s reservoir representation and its corresponding true label; represents the consequent parameter of the fuzzy rule applicable to the target domain reservoir representation; Represents preset hyperparameters.

[0045] The present invention also provides a multi-source domain human posture recognition migration model construction device, comprising:

[0046] A reservoir representation generation module is used to generate a reservoir representation of each source domain and a reservoir representation of the target domain based on multiple source domain data and target domain data of human body posture using a reservoir transformation formula;

[0047] The first similar source domain selection and enhancement module is used to calculate the RSD distance between the reserve pool representation of each source domain and the reserve pool representation of the target domain, select the reserve pool representation of the source domain with the smallest RSD distance with the reserve pool representation of the target domain as the reserve pool representation of the first similar source domain, and use the mix-up formula to enhance the reserve pool representation of the first similar source domain; the RSD distance between the enhanced reserve pool representation of the first similar source domain and the reserve pool representation of the target domain is used as the reference RSD distance, and the remaining source domains except the first similar source domain are used as the candidate source domain set;

[0048] The second similar source domain selection and enhancement module is used to obtain the mix-up ratio range of the reserve pool representation of each source domain in the selected source domain set, with the goal that the RSD distance between the enhanced reserve pool representation of each source domain in the selected source domain set and the reserve pool representation of the target domain is less than the reference RSD distance; the reserve pool representation of the source domain in the selected source domain set whose mix-up ratio range intersects with the preset ratio range is used as the reserve pool representation of the second similar source domain; based on the mix-up ratio range of the reserve pool representation of the second similar source domain, the reserve pool representation of the second similar source domain is enhanced using a mix-up formula;

[0049] The initial model generation module is used to merge the enhanced reserve pool representations of the first and second similar source domains as the reserve pool representation of the most similar source domain; the reserve pool representation of the most similar source domain is input into the TSK fuzzy system to obtain an initial human posture recognition model;

[0050] The optimization model module is used to adjust the consequent parameters of the initial fuzzy rules in the initial human posture recognition model according to the reserve pool representation of the target domain to obtain an optimized human posture recognition model.

[0051] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0052] The method described in the present invention is based on the idea of ​​reserve pool calculation, establishes a time-series dependency relationship between multiple insufficient human posture source domain data and target domain data, and generates reserve pool representations of the source domain and the target domain accordingly; in the process of generating and training a human posture recognition model, the input weights and reserve pool weights do not need to be changed after initialization, which improves the training efficiency compared to traditional recurrent neural networks and recurrent neural network-like methods, and realizes rapid training of the model.

[0053] Secondly, the RSD distance between the pool representations of each source domain and the target domain is calculated, and the pool representation of the source domain with the smallest RSD distance with the pool representation of the target domain is selected as the pool representation of the first similar source domain and enhanced; the pool representation of the second similar source domain is selected again and enhanced according to preset conditions, and the two selection results are merged as the pool representation of the most similar source domain; the similarity between the pool representation of the most similar source domain and the pool representation of the target domain is enhanced, which better solves the problem of changes in the similarity order between each source domain and the target domain due to the arbitrariness of the pool transformation, and improves the accuracy of the learned human posture recognition knowledge.

[0054] Then, the pool representation of the most similar source domain is input into the TSK fuzzy system training to obtain the initial human posture recognition model, the initial human posture recognition model is improved, and a fuzzy classifier based on transfer learning is constructed. The subsequent of the fuzzy rules in the initial human posture recognition model is adjusted in combination with the pool representation of the target domain to obtain the optimized human posture recognition model, which further strengthens the human posture recognition knowledge learned from the source domain and the migration of this knowledge to new tasks. At the same time, the problem of weakened discrimination between source domains of different categories due to sharing the same input weights and pool weights between source domains of different categories during the pool transformation is solved, and the learning efficiency and performance of the model when facing new human posture recognition tasks are improved. Finally, the method of the present invention well ensures that the data in the source domain will not be leaked, and shows good classification performance in actual human posture recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0056] Figure 1 It is the overall flow chart of the method of the present invention;

[0057] Figure 2 It is the overall model architecture of a multi-source domain human posture recognition migration model construction method of the present invention;

[0058] Figure 3 It is a method framework for selecting and enhancing source domains in a method for constructing a multi-source domain human posture recognition migration model of the present invention;

[0059] Figure 4 It is a model verification flow chart of a method for constructing a multi-source domain human posture recognition migration model of the present invention;

[0060] Figure 5 This is Example 2 of the specific implementation of the present invention. Taking HAR10 as an example, the test accuracy is analyzed as K and Graph of changes in . DETAILED DESCRIPTION

[0061] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0062] Example 1

[0063] Figure 1 FIG. 1 is a general flow chart of a method for constructing a multi-source domain human posture recognition migration model according to the present invention. Figure 2 The figure shows the overall model architecture of a multi-source domain human posture recognition migration model construction method provided by the present invention, and the method specifically comprises the following steps:

[0064] S1: Based on multiple source domain data and target domain data of human body posture, the LI-ESN pool transformation formula is used to generate the pool representation of each source domain and the pool representation of the target domain respectively;

[0065] Among them, the input weight matrix is ​​arbitrarily initialized and the reserve pool weight matrix :

[0066] ,

[0067] in, is the state of the reserve pool at time t, is the external input of the system at time t, tanh is the hyperbolic tangent function, is the leakage rate, between 0 and 1.

[0068] S2: Calculate the RSD distance between the pool representation of each source domain and the pool representation of the target domain, select the pool representation of the source domain with the smallest RSD distance with the pool representation of the target domain as the pool representation of the first similar source domain, and use the mix-up formula to enhance the pool representation of the first similar source domain; use the RSD distance between the enhanced pool representation of the first similar source domain and the pool representation of the target domain as the reference RSD distance, and use the remaining source domains except the first similar source domain as the candidate source domain set;

[0069] ① For the reserve pool of the s-th source domain, , estimate the source domain density a through kernel density, and calculate the Stein score function of the reserve pool representation of the source domain: ,

[0070] in, is the number of samples in the source domain, is the reservoir representation of the i-th sample in the source domain, and Represent the kernel function and its bandwidth respectively;

[0071] ② Use the following equation to calculate the reservoir representation of each source domain: and the target domain’s reservoir representation The RSD distance between:

[0072] ,

[0073] ,

[0074] R S D ( H s , H T ) =  h , h ' ∼ H T [ u s ( h , h ') ]  h , h ' ∼ H T [ g ( h , h ') ] ,

[0075] in, and Respectively represent A pool representation of the source domain and a pool representation of the target domain; represents the RSD distance between the reservoir pool representation of the sth source domain and the reservoir pool representation of the target domain; Indicates about The mathematical expectation of is the first sample sampled from the reserve pool representation of the target domain; is a second sample sampled from the reserve pool representation of the target domain; Indicates s Stein scoring function for the pool representation of source domains; is the source domain density; and Respectively represent The number of samples in the source domain and the The source domain A reserve pool is represented; and are the kernel function and its bandwidth respectively; Representation function right The gradient of Representation function right The gradient of Representation function right The gradient of trace It means finding the trace of a matrix; express right The gradient of Indicates transpose.

[0076] ③ Select the source domain’s pool representation with the smallest RSD distance to the target domain’s pool representation as the first similar source domain’s pool representation ;

[0077] ④ Use the following mix-up formula to enhance the pool representation of the first similar source domain:

[0078] ,

[0079] in, and They are the pool representation sampled from the first and second similar source domains and the unique-hot encoding of the corresponding real sample labels. and are the one-hot encodings of the pool representation sampled from the target domain and the corresponding true sample labels, represents the enhanced source domain reservoir representation, represents the sample labels of the enhanced source domain’s reserve pool representation, Represents the mix-up ratio when enhancing the first similar source domain. Random sampling from 0-1; when enhancing the second similar source domain, Randomly sample from the intersection of the mix-up ratio range of the second most similar source domain and 0-1.

[0080] ⑤ Representation of the reserve pool of the first similar source domain enhanced by kernel density estimation The source domain density is recalculated to represent the reserve pool of the first similar source domain. Stein scoring function;

[0081] ⑥ Recalculate the enhanced first similar source domain reserve pool representation and the target domain’s reservoir representation The distance between.

[0082] S3: With the goal that the RSD distance between the enhanced pool representation of each source domain in the candidate source domain set and the pool representation of the target domain is less than the reference RSD distance, the mix-up ratio range of the pool representation of each source domain in the candidate source domain set is obtained respectively; the pool representation of the source domain in the candidate source domain set whose mix-up ratio range intersects with the preset ratio range is used as the pool representation of the second similar source domain; based on the mix-up ratio range of the pool representation of the second similar source domain, the mix-up formula is used to enhance the pool representation of the second similar source domain;

[0083] ① The calculation formula of the RSD distance between the enhanced reserve pool representation of each source domain in the selected source domain set and the reserve pool representation of the target domain is:

[0084] ,

[0085] ,

[0086] ,

[0087] in, Indicates the first s The RSD distance between the enhanced reservoir representation of the source domain and the reservoir representation of the target domain; Indicates the first s The reserve pool representation after the source domain is enhanced; A reservoir representation representing the target domain; Represents the reserve pool representation of the s-th source domain in the candidate source domain set; Indicates s The RSD distance between the source domain’s reservoir representation and the target domain’s reservoir representation; Indicates the first s The reserve pool of source domains represents the mix-up ratio for enhancement; Indicates about The mathematical expectation of is the first sample sampled from the reserve pool representation of the target domain; , The size is The jth column of the identity matrix, m is the dimension represented by the reservoir; Indicates s Stein scoring function for the pool representation of source domains; represents the source domain density; Indicates s The number of samples in the source domain; and are the kernel function and its bandwidth respectively; Indicates s The source domain A reserve pool is represented; The target domain A reserve pool is represented; express right The gradient of represents transpose;

[0088] The RSD distance between the enhanced pool representation of each source domain and the pool representation of the target domain is less than the reference RSD distance, and the target representation is:

[0089] ,

[0090] in, represents the enhanced reserve pool representation of the first similar source domain; Represents the RSD distance between the enhanced reservoir representation of the first similar source domain and the reservoir representation of the target domain.

[0091] ② The preset ratio range is [0,1], and the reserve pool representation of the source domain whose mix-up ratio range in the candidate source domain set intersects with [0,1] is used as the reserve pool representation of the second similar source domain.

[0092] ③ Combine the mix-up ratio range of the reservoir representation of the solved second similar source domain and use the mix-up formula as described in S2 to enhance the reservoir representation of the second similar source domain.

[0093] S5: The enhanced pool representations of the first and second similar source domains are merged as the pool representation of the most similar source domain; the pool representation of the most similar source domain is input into the TSK fuzzy system to obtain the initial human posture recognition model:

[0094] like Figure 3 The figure shows the source domain selection and enhancement method architecture of a multi-source domain human posture recognition migration model construction method provided by the present invention; the most similar source domain reserve pool representation is input into the TSK fuzzy system to obtain the antecedent parameters of the fuzzy rules in the initial human posture recognition model and post-parameters ,The initial human posture recognition model uses the Epanechnikov kernel function as the membership function.

[0095] S6: adjusting the consequent parameters of the initial fuzzy rules in the initial human posture recognition model according to the reserve pool representation of the target domain to obtain an optimized human posture recognition model.

[0096] ① Fuzzy target domain reserve pool representation , the empirical risk term on the reserve pool representation of the fuzzified target domain and the knowledge regularization term of the historical consequent parameter migration are used as the objective function to calculate the consequent parameters corresponding to the adjusted fuzzy rules. The objective function expression is as follows: ,

[0097] in, It is the reserve pool representation of the target domain after the fuzzification operation; Represents the one-hot encoding of the true labels corresponding to all samples in the target domain; are the consequent parameters of the fuzzy rules trained on the selected most similar source domain’s reservoir representation and its corresponding true label; The consequent parameters of the fuzzy rules represented by the reservoir that are applicable to the target domain; Represents a preset hyperparameter that can be set through cross-validation.

[0098] ②The solution formula of the objective function is as follows:

[0099] ,

[0100] in, It is the reserve pool representation of the target domain after the fuzzification operation; represents the transpose of the reservoir representation of the target domain after the fuzzification operation; Represents the one-hot encoding of the true labels corresponding to all samples in the target domain; Represents the transpose of the one-hot encoding of the true labels corresponding to all samples in the target domain; Indicates size The identity matrix of are the consequent parameters of the fuzzy rules trained on the selected most similar source domain’s reservoir representation and its corresponding true label; represents the consequent parameter of the fuzzy rule applicable to the target domain reservoir representation; Represents preset hyperparameters.

[0101] The human body posture data to be measured is input into the optimized human body posture recognition model to obtain the recognition result of the human body posture data to be measured.

[0102] Example 2

[0103] Based on Example 1, a multi-source domain human posture recognition migration model construction method of the present invention can be applied to various scenarios such as medical health monitoring and intelligent transportation systems.

[0104] ① In the field of medical health, real-time monitoring and early warning systems are of great significance for the prevention and management of diseases. Especially for chronic diseases such as diabetes, hypertension, and heart disease, real-time analysis of time series data can provide important health trend information. However, medical data usually comes from multiple hospitals or regions, and these data sources may have insufficient problems, such as missing data and inconsistent collection cycles. For example, a hospital hopes to establish a real-time health monitoring system to monitor the blood sugar levels of diabetic patients and promptly warn of possible health risks. Due to the small data sample of the hospital, it cannot train a high-performance prediction model alone, but it can apply large-scale data from multiple other hospitals or research institutions to its own system through transfer learning.

[0105] In this application scenario, the multi-source domain human posture recognition migration model construction method described in the present invention can process blood glucose data from patients in multiple hospitals in real time and identify possible abnormal patterns. For example, when an abnormal fluctuation in a patient's blood glucose level is detected, the model can classify and issue an early warning in a timely manner, reminding doctors and patients to take measures to prevent potential acute complications or long-term health problems. The model can also learn the health trends of different patient groups from data from other hospitals and migrate this information to the target hospital to improve the accuracy and real-time nature of the prediction. In this way, the medical health monitoring system can still provide efficient real-time monitoring and early warning in the absence of sufficient data, improve the health management effect of patients, and reduce the burden on medical resources.

[0106] ② Traffic flow prediction and control in intelligent transportation systems is a complex and important task. Urban traffic data usually comes from multiple insufficient source domains, such as data from different cities or different time periods. These data are highly dependent in time and space, but the insufficiency and inconsistency of the data increase the difficulty of prediction and control. Specifically, suppose that a city's traffic management department wants to optimize the traffic signal control system to alleviate traffic congestion during peak hours. The city may lack sufficient historical traffic data, but it can learn from the traffic data of other cities. Through the method for constructing a multi-insufficient source domain human posture recognition model for knowledge transfer described in the present invention, the traffic patterns of multiple other cities can be migrated to the target city, thereby realizing real-time traffic flow prediction and timely adjusting the timing plan of traffic lights to alleviate traffic pressure. This intelligent transportation system based on time series migration of multiple insufficient source domains can significantly improve the efficiency of traffic management, reduce traffic congestion time, reduce the waiting time of travelers, and improve the operating efficiency and safety of the overall transportation system.

[0107] The present invention uses the HumanActivityRecongnition dataset and MotionSense dataset on the UCI website as experimental datasets. By using a smart phone with embedded inertial sensors, these datasets include physical signals of volunteers performing 6 different daily activities, such as gravity, acceleration, and rotation rate. In order to simulate multiple scenarios with insufficient source domains and target domains, data from different volunteers are used as different domains in the experiment.

[0108] In the experiment, 30 samples were randomly selected from each volunteer as the training set, and the rest were used as the test set. Five-fold cross validation was used to determine the parameters. The detailed information of the data set in the experiment is recorded in Table 1:

[0109] Table 1

[0110]

[0111] Figure 4 This is a model verification flow chart of a method for constructing a multi-source domain human posture recognition migration model of the present invention; the specific experimental steps include:

[0112] (1) Data division. The entire dataset is divided into different domains according to different volunteer numbers. 30 samples are selected from each domain as the training set, and the remaining samples are used as the test set. Then, 14 datasets are constructed by combining the volunteer numbers in the table above.

[0113] (2) Model parameter setting. In the reservoir transformation part, the number of reservoir units, the sparsity rate, the leakage rate and the spectrum radius are set to 30, 0.05, 0.2 and 0.9 respectively. In the source domain selection and mix-up ratio solution part, the Epanechnikov kernel function is used.

[0114] (3) Model selection. Since the model will have different effect deviations for a specific data set, grid optimization is required. The main purpose is to obtain the best parameters for a specific model for a specific data set. The optimal parameters are found from the parameter optimization range by a five-fold cross-validation method. The current optimal parameters are continuously updated until the iteration ends to obtain the final optimal hyperparameters. In this process, the model results are observed through multiple experiments, combined with the performance of the model on the validation set, to determine whether the model has achieved the best fit, and gradually determine the optimal optimization range of the model parameters. The algorithm proposed in the present invention needs to target the number of rules K and the trade-off coefficient Two hyperparameters are optimized, the optimization range of the number of rules K is 6-20, and the weight coefficient The optimization range is [ 2 − 10 , 2 − 9 ,..., 2 10 ] .

[0115] (4) Retraining: Use grid optimization to find the optimal hyperparameters to retrain the model.

[0116] (5) Model testing: Use the test set to test the trained model and observe whether the test results meet expectations.

[0117] (6) Evaluation indicators. The test results of the test set have been obtained through the above steps, and the accuracy is calculated using the formula.

[0118] This embodiment compares a multi-source domain human posture recognition transfer model construction method provided by the present invention with existing classic machine learning algorithms. The comparison algorithms include: Leaky Integrator Echo State Network (LI-ESN), Weighted Average Ensemble for Transfer Learning (WAETL), Co-tuning for transfer learning (Co-tuning), and Fuzzy Multiple Source Transfer Learning (FMS-TL, TSK-FS), which are trained only on the source domain and target domain respectively. The mean and variance of the accuracy of five experiments of these algorithms are recorded as shown in Table 2 (all results are retained to 4 decimal places for easy observation of the results):

[0119] Table 2

[0120]

[0121] In addition, considering the hyperparameters K and The sensitivity of the algorithm of the present invention is analyzed by taking HAR10 as an example to analyze the test accuracy with K and Refer to the changes in Figure 5 As shown by Figure 5 It can be seen that as K increases, the test accuracy of the algorithm of the present invention increases slightly first and then remains stable, which shows that in practical applications, K does not need to be set too large, and only needs to be slightly larger than the number of categories. The test accuracy of the algorithm of the present invention first increases, then stabilizes, and finally decreases. This shows that in practical applications It is necessary to perform careful optimization. As can be seen from the table, in terms of classification accuracy, the multi-source domain human posture recognition transfer model construction method provided by the present invention has better performance than other algorithms.

[0122] Based on the multi-source domain human posture recognition migration model construction method provided in the above embodiment, the embodiment of the present application also provides a multi-source domain human posture recognition migration model construction device, which specifically includes:

[0123] A reservoir representation generation module is used to generate a reservoir representation of each source domain and a reservoir representation of the target domain based on multiple source domain data and target domain data of human body posture using a reservoir transformation formula;

[0124] The first similar source domain selection and enhancement module is used to calculate the RSD distance between the reserve pool representation of each source domain and the reserve pool representation of the target domain, select the reserve pool representation of the source domain with the smallest RSD distance with the reserve pool representation of the target domain as the reserve pool representation of the first similar source domain, and use the mix-up formula to enhance the reserve pool representation of the first similar source domain; the RSD distance between the enhanced reserve pool representation of the first similar source domain and the reserve pool representation of the target domain is used as the reference RSD distance, and the remaining source domains except the first similar source domain are used as the candidate source domain set;

[0125] The second similar source domain selection and enhancement module is used to obtain the mix-up ratio range of the reserve pool representation of each source domain in the selected source domain set, with the goal that the RSD distance between the enhanced reserve pool representation of each source domain in the selected source domain set and the reserve pool representation of the target domain is less than the reference RSD distance; the reserve pool representation of the source domain in the selected source domain set whose mix-up ratio range intersects with the preset ratio range is used as the reserve pool representation of the second similar source domain; based on the mix-up ratio range of the reserve pool representation of the second similar source domain, the reserve pool representation of the second similar source domain is enhanced using a mix-up formula;

[0126] The initial model generation module is used to merge the enhanced reserve pool representations of the first and second similar source domains as the reserve pool representation of the most similar source domain; the reserve pool representation of the most similar source domain is input into the TSK fuzzy system to obtain an initial human posture recognition model;

[0127] The optimization model module is used to adjust the consequent parameters of the initial fuzzy rules in the initial human posture recognition model according to the reserve pool representation of the target domain to obtain an optimized human posture recognition model.

[0128] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A method for constructing a multi-source domain human posture recognition migration model, characterized in that: The following steps are involved: Based on multiple source domain data and target domain data of human body posture, the reservoir pool representation of each source domain and the reservoir pool representation of the target domain are generated respectively by using the reservoir pool transformation formula; Calculate the RSD distance between the pool representation of each source domain and the pool representation of the target domain, select the pool representation of the source domain with the smallest RSD distance with the pool representation of the target domain as the pool representation of the first similar source domain, and use the mix-up formula to enhance the pool representation of the first similar source domain; use the RSD distance between the enhanced pool representation of the first similar source domain and the pool representation of the target domain as the reference RSD distance, and use the remaining source domains except the first similar source domain as the candidate source domain set; Taking the RSD distance between the enhanced pool representation of each source domain in the selected source domain set and the pool representation of the target domain as less than the reference RSD distance as the goal, respectively obtain the mix-up ratio range of the pool representation of each source domain in the selected source domain set; take the pool representation of the source domain in the selected source domain set where the mix-up ratio range intersects with the preset ratio range as the pool representation of the second similar source domain; based on the mix-up ratio range of the pool representation of the second similar source domain, use the mix-up formula to enhance the pool representation of the second similar source domain; wherein the mix-up formula is specifically expressed as follows: ， in, and They are the pool representation sampled from the first and second similar source domains and the one-hot encoding of the corresponding real sample labels; and They are the one-hot encodings of the pool representation sampled from the target domain and the corresponding true sample labels; Reservoir representation representing the enhanced source domain; The sample labels of the reserve pool representation of the enhanced source domain; Represents the mix-up ratio when enhancing the first similar source domain. Random sampling from 0-1; when enhancing the second similar source domain, Randomly sample from the intersection of the mix-up ratio range of the second similar source domain and 0-1; The enhanced pool representations of the first and second similar source domains are merged as the pool representation of the most similar source domain; the pool representation of the most similar source domain is input into the TSK fuzzy system to obtain an initial human posture recognition model; The consequent parameters of the initial fuzzy rules in the initial human posture recognition model are adjusted according to the reserve pool representation of the target domain to obtain an optimized human posture recognition model.

2. The method for constructing a multi-source domain human posture recognition migration model according to claim 1, characterized in that: The calculating of the RSD distance between the reservoir pool representation of each source domain and the reservoir pool representation of the target domain includes: ， ， ， ， in, and Respectively represent A pool representation of the source domain and a pool representation of the target domain; represents the RSD distance between the reservoir pool representation of the sth source domain and the reservoir pool representation of the target domain; Indicates about The mathematical expectation of is the first sample sampled from the reserve pool representation of the target domain; is a second sample sampled from the reserve pool representation of the target domain; Indicates s Stein scoring function for the pool representation of source domains; is the source domain density; and Respectively represent The number of samples in the source domain and the The source domain A reserve pool is represented; and are the kernel function and its bandwidth respectively; Representation function right The gradient of Representation function right The gradient of Representation function right The gradient of trace It means finding the trace of a matrix; express right The gradient of Indicates transpose.

3. The method for constructing a multi-source domain human posture recognition migration model according to claim 1, characterized in that: The calculation formula of the RSD distance between the enhanced reservoir representation of each source domain in the selected source domain set and the reservoir representation of the target domain is: ， ， ， in, Indicates the first s The RSD distance between the enhanced reservoir representation of the source domain and the reservoir representation of the target domain; Indicates the first s The reserve pool representation after the source domain is enhanced; A reservoir representation representing the target domain; Represents the reserve pool representation of the s-th source domain in the candidate source domain set; Indicates s The RSD distance between the source domain’s reservoir representation and the target domain’s reservoir representation; Indicates the first s The reserve pool of source domains represents the mix-up ratio for enhancement; Indicates about The mathematical expectation of is the first sample sampled from the reserve pool representation of the target domain; , The size is The jth column of the identity matrix, m is the dimension represented by the reservoir; Indicates s Stein scoring function for the pool representation of source domains; represents the source domain density; Indicates s The number of samples in the source domain; and are the kernel function and its bandwidth respectively; Indicates s The source domain A reserve pool is represented; The target domain A reserve pool is represented; express right The gradient of represents transpose; The RSD distance between the enhanced pool representation of each source domain and the pool representation of the target domain is less than the reference RSD distance, and the target representation is: ， in, represents the enhanced reserve pool representation of the first similar source domain; Represents the RSD distance between the enhanced reservoir representation of the first similar source domain and the reservoir representation of the target domain.

4. The method for constructing a multi-source domain human posture recognition migration model according to claim 1, characterized in that: The preset ratio range is [0,1].

5. The method for constructing a multi-source domain human posture recognition migration model according to claim 1, characterized in that: The initial fuzzy rules are expressed as follows: ， ， ， in, Indicates k Under the fuzzy rule m dimensional features, defined by a membership function; is the first sample sampled from the reserve pool representation of the target domain; express The m-th dimension feature of is the cth first-order linear output under the kth fuzzy rule; It is k Under the fuzzy rule c The output j coefficients; C Represents the number of first-order linear outputs of the fuzzy rule.

6. The method for constructing a multi-source domain human posture recognition migration model according to claim 1, characterized in that: The adjusting the consequent parameters of the initial fuzzy rules in the initial human posture recognition model according to the reserve pool representation of the target domain specifically includes: The empirical risk term represented by the reserve pool of the target domain and the knowledge regularization term of the historical consequent parameter migration are used as the objective function to solve the objective function; The consequent parameters of the initial fuzzy rules in the initial human posture recognition model are adjusted according to the solution of the objective function.

7. The method for constructing a multi-source domain human posture recognition migration model according to claim 6, characterized in that: The objective function expression is as follows: ， in, It is the reserve pool representation of the target domain after the fuzzification operation; Represents the one-hot encoding of the true labels corresponding to all samples in the target domain; are the consequent parameters of the fuzzy rules trained on the selected most similar source domain’s reservoir representation and its corresponding true label; The consequent parameters of the fuzzy rules represented by the reservoir that are applicable to the target domain; Represents preset hyperparameters.

8. The method for constructing a multi-source domain human posture recognition migration model according to claim 6, characterized in that: The solution formula of the objective function is as follows: ， in, It is the reserve pool representation of the target domain after the fuzzification operation; represents the transpose of the reservoir representation of the target domain after the fuzzification operation; Represents the one-hot encoding of the true labels corresponding to all samples in the target domain; Represents the transpose of the one-hot encoding of the true labels corresponding to all samples in the target domain; Indicates size The identity matrix of are the consequent parameters of the fuzzy rules trained on the selected most similar source domain’s reservoir representation and its corresponding true label; represents the consequent parameter of the fuzzy rule applicable to the target domain reservoir representation; Represents preset hyperparameters.

9. A multi-source domain human posture recognition migration model construction device, characterized in that: include: A reservoir representation generation module is used to generate a reservoir representation of each source domain and a reservoir representation of the target domain based on multiple source domain data and target domain data of human body posture using a reservoir transformation formula; The first similar source domain selection and enhancement module is used to calculate the RSD distance between the reserve pool representation of each source domain and the reserve pool representation of the target domain, select the reserve pool representation of the source domain with the smallest RSD distance with the reserve pool representation of the target domain as the reserve pool representation of the first similar source domain, and use the mix-up formula to enhance the reserve pool representation of the first similar source domain; the RSD distance between the enhanced reserve pool representation of the first similar source domain and the reserve pool representation of the target domain is used as the reference RSD distance, and the remaining source domains except the first similar source domain are used as the candidate source domain set; The second similar source domain selection and enhancement module is used to obtain the mix-up ratio range of the reserve pool representation of each source domain in the selected source domain set, respectively, with the goal that the RSD distance between the enhanced reserve pool representation of each source domain in the selected source domain set and the reserve pool representation of the target domain is less than the reference RSD distance; the reserve pool representation of the source domain in the selected source domain set whose mix-up ratio range intersects with the preset ratio range is used as the reserve pool representation of the second similar source domain; based on the mix-up ratio range of the reserve pool representation of the second similar source domain, the reserve pool representation of the second similar source domain is enhanced using the mix-up formula; wherein the specific expression of the mix-up formula is as follows: ， in, and They are the pool representation sampled from the first and second similar source domains and the one-hot encoding of the corresponding real sample labels; and They are the one-hot encodings of the pool representation sampled from the target domain and the corresponding true sample labels; Reservoir representation representing the enhanced source domain; The sample labels of the reserve pool representation of the enhanced source domain; Represents the mix-up ratio when enhancing the first similar source domain. Random sampling from 0-1; when enhancing the second similar source domain, Randomly sample from the intersection of the mix-up ratio range of the second similar source domain and 0-1; The initial model generation module is used to merge the enhanced reserve pool representations of the first and second similar source domains as the reserve pool representation of the most similar source domain; the reserve pool representation of the most similar source domain is input into the TSK fuzzy system to obtain an initial human posture recognition model; The optimization model module is used to adjust the consequent parameters of the initial fuzzy rules in the initial human posture recognition model according to the reserve pool representation of the target domain to obtain an optimized human posture recognition model.

Images