Behavior recognition method and terminal under multiple usage modes based on generative adversarial network

Abstract

The present invention discloses a method and terminal for behavior recognition under multiple usage modes based on a generative adversarial network. The method comprises: collecting behavior data under multiple usage modes to obtain a source domain dataset and a target domain dataset, and using the obtained source domain dataset and target domain dataset as inputs to a GAN model; extracting domain-invariant representations shared across domains between the target and source domains through the GAN model, and generating a synthetic sequence in the target domain based on the label sequence of the source domain dataset; performing transfer learning based on the synthetic sequence generated in the target domain, and predicting behavior labels under multiple usage modes based on the learned domain-invariant representations. The present invention solves the problem that existing behavior recognition technologies require collecting a large amount of sensor data in new environments to calculate an accurate transformation matrix, thereby reducing the workload of data collection and speeding up data processing.

Description

Technical Field

[0001] The present invention relates to the field of indoor positioning technology, and in particular to a behavior recognition method, terminal, and storage medium under multiple usage modes based on a generative adversarial network. Background Art

[0002] In recent years, indoor positioning has become a major research hotspot in the field of location-based services. Smartphone-based indoor positioning eliminates the need for users to carry additional equipment, facilitating the widespread adoption of indoor positioning technology. In indoor environments, human activity contains rich semantic information. If a user's activity is identified as taking an elevator, their location can be inferred. These activities can serve as landmarks for indoor positioning and mapping.

[0003] In recent years, the concept of deep learning has garnered considerable attention. Deep learning-based applications are numerous, including image processing, intelligent transportation systems, and indoor positioning. Numerous studies have confirmed that deep learning models possess superior feature representation capabilities, making them more effective at handling complex classification tasks. This is because deep learning methods integrate feature extraction and classification with neural networks, enabling the automatic learning of appropriate features. Therefore, deep learning-based behavior recognition methods hold great promise for future applications.

[0004] Traditional behavior recognition methods primarily rely on wearable sensors, whose posture is less affected by user usage patterns. However, the greatest challenge for implementing behavior recognition methods based on smart terminals is addressing the impact of posture on multiple user patterns. Furthermore, current deep learning-based behavior recognition methods still require extensive training data to ensure accuracy, consuming significant manpower and material resources, which limits their widespread adoption and application.

[0005] Therefore, the existing technology needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that, in response to the defects of the existing technology, the present invention provides a behavior recognition method and terminal under multiple usage modes based on a generative adversarial network to solve the problem that the existing behavior recognition method is time-consuming and labor-intensive, while reducing the impact of posture differences under different usage modes on the accuracy of behavior recognition, and realizing behavior recognition under multiple usage modes.

[0007] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0008] In a first aspect, the present invention provides a method for behavior recognition under multiple usage modes based on a generative adversarial network, comprising:

[0009] Collect behavioral data under multiple usage modes to obtain source domain datasets and target domain datasets, and use the obtained source domain datasets and target domain datasets as inputs to the GAN model;

[0010] Extracting domain-invariant representations shared across domains between the target domain and the source domain through a GAN model, and generating synthetic sequences in the target domain based on the labeled sequence of the source domain dataset;

[0011] Synthetic sequences are generated according to the target domain for transfer learning, and behavioral labels under multiple usage modes are predicted based on the learned domain-invariant representations.

[0012] In one implementation, collecting behavioral data in multiple usage modes to obtain a source domain dataset and a target domain dataset includes:

[0013] Collecting behavioral data in handheld mode, using the behavioral data in handheld mode as a source domain dataset, and labeling corresponding behaviors in the source domain dataset;

[0014] Behavioral data of other usage patterns are collected and used as target domain data.

[0015] In one implementation, the GAN model includes: an RNN sequence encoder, a GAN generator, an RNN reconstruction decoder, and a behavior predictor shared by all domains.

[0016] In one implementation, extracting domain-invariant representations shared across domains between the target domain and the source domain through a GAN model includes:

[0017] The domain-invariant representation shared across domains between the target domain and the source domain is extracted through the RNN sequence encoder and the specific domain vector:

[0018] The common feature information extracted from the sensor signals generated according to different behaviors in different usage modes is analyzed, different domain vectors are used in the source domain and the target domain to distinguish the specific features of each domain, and preset function parameters are used to share them between the source domain and the target domain.

[0019] In one implementation, generating a synthetic sequence in the target domain based on a labeled sequence of a source domain dataset includes:

[0020] Combining the GAN generator with the RNN sequence encoder to obtain a sequence conversion model;

[0021] A synthetic sequence is generated in the target domain according to the sequence conversion model and the labeled sequence of the source domain dataset.

[0022] In one implementation, generating a synthetic sequence in the target domain based on the sequence conversion model and the labeled sequence of the source domain dataset then includes:

[0023] According to cycle consistency regularization, the sequence generated by the source domain is mapped, and an improved generated synthetic sequence is obtained by defining a consistency loss penalty optimizer;

[0024] According to perceptual consistency regularization, the RNN sequence encoder is used as a perceptual function to perform semantic representation constants after the GAN generator is converted to the target domain.

[0025] In one implementation, the step of generating a synthetic sequence in the target domain based on the sequence conversion model and the labeled sequence of the source domain dataset further includes:

[0026] The synthetic sequence is reconstructed by the RNN reconstruction decoder: reconstruction is performed according to the domain invariant representation, the reconstruction loss function and the noise obeying the normal distribution.

[0027] In one implementation, generating a synthetic sequence based on the target domain for transfer learning, and predicting behavior labels under multiple usage modes based on the learned domain-invariant representation, includes:

[0028] Optimizing domain-invariant representations through the GAN generator and the RNN reconstruction decoder;

[0029] When the extracted domain-invariant representation meets the requirements, the content learned in the source domain is transferred to the target domain through the behavior predictor shared by all domains;

[0030] Predicting behavior labels in multiple usage modes based on the learned domain-invariant representation.

[0031] In a second aspect, the present invention also provides a computer terminal comprising: a processor and a memory, wherein the memory stores a behavior recognition program under multiple usage modes based on a generative adversarial network, and when the behavior recognition program under multiple usage modes based on a generative adversarial network is executed by the processor, it is used to implement the operation of the behavior recognition method under multiple usage modes based on a generative adversarial network as described in the first aspect.

[0032] In a third aspect, the present invention also provides a computer-readable storage medium, which stores a behavior recognition program under multiple usage modes based on a generative adversarial network. When the behavior recognition program under multiple usage modes based on a generative adversarial network is executed by a processor, it is used to implement the operation of the behavior recognition method under multiple usage modes based on a generative adversarial network as described in the first aspect.

[0033] The present invention adopts the above technical solution to achieve the following effects:

[0034] By collecting behavioral data under multiple usage modes, the present invention can obtain source domain datasets and target domain datasets, and use the obtained source domain datasets and target domain datasets as inputs to the GAN model. Furthermore, the GAN model can extract domain-invariant representations shared across the target and source domains, and can generate synthetic sequences in the target domain based on the labeled sequence of the source domain dataset. Transfer learning can then be performed based on the synthetic sequences generated in the target domain, and behavioral labels under multiple usage modes can be predicted based on the learned domain-invariant representations. This invention solves the problem that existing behavior recognition technologies require collecting a large amount of sensor data from new environments to calculate an accurate transformation matrix, thereby reducing the workload of data collection and accelerating data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0036] Figure 1 This is a flowchart of a method for behavior recognition under multiple usage modes based on a generative adversarial network in one implementation of the present invention.

[0037] Figure 2 This is a schematic diagram of a framework for behavior recognition under multiple usage modes based on a generative adversarial network in one implementation of the present invention.

[0038] Figure 3 It is a schematic diagram of data collection in one implementation of the present invention.

[0039] Figure 4 It is a schematic diagram of the data format collected in one implementation of the present invention.

[0040] Figure 5 It is a schematic diagram of a sensor sequence in one implementation of the present invention.

[0041] Figure 6 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0042] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] Exemplary Methods

[0045] Traditional behavior recognition methods primarily rely on wearable sensors, whose posture is less affected by user usage patterns. However, the greatest challenge for implementing behavior recognition methods based on smart terminals is addressing the impact of posture on multiple user patterns. Furthermore, current deep learning-based behavior recognition methods still require extensive training data to ensure accuracy, consuming significant manpower and material resources, which limits their widespread adoption and application.

[0046] In response to the above technical problems, an embodiment of the present invention provides a behavior recognition method under multiple usage modes based on a generative adversarial network. The embodiment of the present invention solves the problem that existing behavior recognition technology needs to collect more sensor data in new environments to calculate an accurate transformation matrix, thereby reducing the workload of data collection and speeding up data processing.

[0047] like Figure 1 As shown, an embodiment of the present invention provides a method for behavior recognition under multiple usage modes based on a generative adversarial network, comprising the following steps:

[0048] Step S100 , collecting behavioral data in multiple usage modes, obtaining a source domain dataset and a target domain dataset, and using the obtained source domain dataset and target domain dataset as inputs of a GAN model.

[0049] This embodiment provides a behavior recognition method based on a generative adversarial network (GAN). This method uses IMU sensor data from smart devices (e.g., smartphones, tablets, etc.) for behavior recognition. The method divides the samples into a source domain dataset and a target domain dataset. Furthermore, the source domain dataset only contains IMU data from a single usage mode, requiring only labeling, reducing the workload of labeling data from other usage modes. After acquiring the sample dataset, a joint learning process using a GAN generator and an RNN reconstruction decoder module is performed to extract more common feature information from samples with different usage modes. This reduces the impact of posture differences on model predictions and reduces the sample size while ensuring accuracy.

[0050] Specifically, in one implementation of the embodiment of the present invention, step S100 includes the following steps:

[0051] Step S101: collecting behavioral data in handheld mode, using the behavioral data in handheld mode as a source domain dataset, and labeling corresponding behaviors in the source domain dataset;

[0052] Step S102 : collecting behavior data of other usage patterns, and using the behavior data of other usage patterns as target domain data.

[0053] In this embodiment, in order to solve the problem that existing behavior recognition methods are time-consuming and labor-intensive, and at the same time reduce the impact of posture differences in different usage modes on behavior recognition accuracy, behavior recognition in multiple usage modes is achieved.

[0054] In this embodiment, you can use the self-developed Android APP (such as Figure 3 As shown), the acceleration data and angular velocity data are collected. The collected data format and the corresponding data are as follows Figure 4 As shown; after obtaining these data, the collected acceleration data and angular velocity data are used as input to the GAN model; the acceleration data and angular velocity data collected in this embodiment are behavioral data in handheld mode and other modes, wherein the behavioral data in the handheld mode include but are not limited to: walking, going up and down elevators, going up and down stairs, and going up and down slopes; the behavioral data in other usage modes include but are not limited to: making phone calls, shaking hands, and putting in pockets and backpacks.

[0055] In this embodiment, the collected behavior data of walking, going up and down elevators, going up and down stairs, and going up and down slopes in handheld mode can be used as the source domain data set, and various behaviors in the source domain data set are labeled to obtain a labeled sequence (such as Figure 5 As shown in the figure); behavioral data collected for other usage modes (for example, making phone calls, pockets, backpacks, and hand-waving) can be used as target domain datasets. The target domain datasets do not need to be labeled, that is, the target domain datasets are unlabeled sequences.

[0056] In this embodiment, a source domain dataset and a target domain dataset are collected, and the collected source domain dataset and the target domain dataset can be input into a generative adversarial network (GAN) model in this embodiment to perform behavior recognition under multiple usage modes.

[0057] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for behavior recognition under multiple usage modes based on a generative adversarial network further includes the following steps:

[0058] Step S200: extracting domain-invariant representations shared across domains between the target domain and the source domain through a GAN model, and generating a synthetic sequence in the target domain according to the labeled sequence of the source domain dataset.

[0059] like Figure 2 As shown, in this embodiment, the GAN model includes: an RNN sequence encoder, a GAN generator, an RNN reconstruction decoder, and a behavior predictor shared by all domains.

[0060] In this embodiment, the GAN model assumes that there are two domains: the source domain and the target domain, where the sequence in the source domain is the labeled sequence ,in , is the triaxial accelerometer data, is the gyroscope data, n is the number of samples, is the behavior label; the sequence of the target domain is the unlabeled sequence The objectives of the model are divided into three parts:

[0061] 1) Extracting domain-invariant representation z that is shared across domains;

[0062] 2) According to Generate in the target domain ;

[0063] 3) Predicting behavior labels in the target domain .

[0064] Specifically, in one implementation of the embodiment of the present invention, step S200 includes the following steps:

[0065] Step S201: extracting a domain-invariant representation shared across domains between the target domain and the source domain through the RNN sequence encoder and a specific domain vector: analyzing the common feature information extracted from sensor signals generated according to different behaviors in different usage modes, using different domain vectors in the source domain and the target domain to distinguish the specific features of each domain, and using preset function parameters to share between the source domain and the target domain.

[0066] First, in this embodiment, an RNN sequence encoder and a specific domain vector are used to extract a domain-invariant representation Z shared across domains. The domain-invariant representation Z is the common feature information extracted by analyzing sensor signals generated according to different behaviors under different usage modes.

[0067] When extracting common feature information, different domain vectors are used in the source and target domains to distinguish the specific features of each domain, and the function The parameters of (i.e., the preset function parameters) are shared between the two domains. The domain-invariant feature Z is defined as follows:

[0068]

[0069] in, is a specific domain vector; For accelerometer and gyroscope data.

[0070] Specifically, in one implementation of the embodiment of the present invention, step S200 includes the following steps:

[0071] Step S202, combining the GAN generator and the RNN sequence encoder to obtain a sequence conversion model;

[0072] Step S203 : generating a synthetic sequence in the target domain according to the sequence conversion model and the labeled sequence of the source domain dataset.

[0073] In this example, a Generative Adversarial Network (GAN) generator is constructed , according to the sequence in the source domain Generate synthetic sequences in the target domain , the synthetic sequence will be re-enhanced in the RNN sequence encoder to extract the domain invariant representation, and by combining it with the RNN sequence encoder, a sequence conversion model can be obtained. , where the synthetic sequence The definition is as follows:

[0074]

[0075] in, is the domain vector specific to the source domain; is the source domain sequence.

[0076] In addition to learning the source domain dataset, the target domain data can also be used to learn domain-invariant representations. Therefore, an inverse mapping model is constructed in this embodiment. , the inverse mapping model can be based on the target domain sequence Generate synthetic columns in the source domain :

[0077]

[0078] in, is the domain vector specific to the target domain; is the target domain sequence.

[0079] The RNN sequence encoder can be based on the synthetic sequence and Strengthen the learning of domain-invariant representation z, where the sequence transformation model and inverse mapping models Both are discriminators for the target domain Training, discriminator The role of is to distinguish synthetic sequences from real sequences. This GAN model promotes the RNN sequence encoder Capturing domain-invariant features of different domains, RNN sequence encoder Depend on and In the target domain GAN generator, the loss function is defined in this embodiment. , define the loss function in the source domain GAN generator , the total loss of GAN is :

[0080]

[0081] in, and are the discriminators for the source and target domains respectively; and are domain vectors specific to the source and target domains, respectively; and are the sequences of the source domain and the target domain respectively.

[0082] Specifically, in one implementation of the embodiment of the present invention, the following steps are included after step S203:

[0083] Step S204: mapping the sequence generated in the source domain according to cycle consistency regularization, and obtaining an improved generated synthetic sequence by defining a consistency loss penalty optimizer;

[0084] Step S205 , according to perceptual consistency regularization, the RNN sequence encoder is used as a perceptual function, and a semantic representation constant is executed after the GAN generator is converted to the target domain.

[0085] In order to improve the generation of synthetic sequences, this embodiment also introduces cycle consistency regularization to ensure that the sequences generated from the source domain can be mapped back, reducing the loss of content information. By defining the L-1 consistency loss To penalize the optimizer, encourage the GAN generator to generate data in meaningful directions:

[0086]

[0087] In order to further standardize the learning of domain-invariant representation, this embodiment introduces perceptual consistency regularization to encode As a perceptron, it enforces semantic representation constancy after the generator is converted to another domain. For example, in the source domain, the encoder and Extracting domain-invariant representations 、 , by calculating 、 The minimum L-2 distance between To encourage immutability:

[0088]

[0089] The domain-invariant representation information extracted by a single GAN generator is not perfect enough. Unlike the traditional GAN ​​model, this embodiment also uses an RNN reconstruction decoder. It is jointly trained with the GAN generator to extract more complete feature information.

[0090] Specifically, in one implementation of the embodiment of the present invention, the following steps are included after step S203:

[0091] Step S206 , reconstructing the synthesized sequence through the RNN reconstruction decoder: reconstructing according to the domain invariant representation, the reconstruction loss function, and the noise that obeys the normal distribution.

[0092] The RNN reconstruction decoder reconstructs the sequence based on the domain-invariant representation z In the RNN reconstruction decoder, this embodiment introduces a noise to the domain invariant representation z To simplify the training process, noise Obeying the normal distribution, the reconstruction process is as follows:

[0093]

[0094] in Domain-invariant representation reconstructed for the decoder; is the noise that obeys the normal distribution; The synthesized sequence reconstructed for the decoder.

[0095] As with the GAN generator, the synthetic sequence We will return to the encoder to continue extracting the domain-invariant representation z; like the encoder, the RNN reconstruction decoder is shared between all domains. In this embodiment, the reconstruction loss function is defined as :

[0096]

[0097] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for behavior recognition under multiple usage modes based on a generative adversarial network further includes the following steps:

[0098] Step S300 : generating a synthetic sequence according to the target domain for transfer learning, and predicting behavior labels under multiple usage modes according to the learned domain-invariant representation.

[0099] In this embodiment, the behavior labels under multiple usage modes are predicted based on the learned domain-invariant representation, and then the behaviors under the corresponding usage modes are identified based on the behavior labels. By identifying these behaviors under multiple indoor usage modes, they can be used as landmarks for indoor positioning and mapping, thereby providing corresponding data for indoor positioning technology.

[0100] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0101] Step S301, optimizing domain-invariant representation through the GAN generator and the RNN reconstruction decoder;

[0102] Step S302 , when the extracted domain-invariant representation meets the requirement, the content learned in the source domain is transferred to the target domain through the behavior predictor shared by all domains;

[0103] Step S303 : predicting behavior labels in multiple usage modes based on the learned domain-invariant representation.

[0104] In this embodiment, the GAN generator and reconstruction decoder are used to continuously optimize the domain-invariant features. When the extracted domain-invariant representation meets the requirements, a behavior predictor shared by all domains can be constructed. Transfer the content learned in the source domain to the target domain to predict the behavior label of the target domain sequence , the prediction method is as follows:

[0105]

[0106] By constructing a joint regression loss function :

[0107]

[0108] Finally, the total loss of the model can be obtained :

[0109]

[0110] in The value of weight needs to be adjusted according to the actual situation.

[0111] It is worth mentioning that in the process of data collection by the IMU sensor in this embodiment, in order to improve the prediction accuracy, based on the three-axis acceleration data and gyroscope data, the influence of air pressure on behavior recognition can also be considered, that is, air pressure is used as an error factor in behavior prediction, and then the prediction accuracy of the entire model is improved by recognition training under different air pressures.

[0112] This embodiment achieves the following technical effects through the above technical solution:

[0113] This embodiment proposes a behavior recognition method under multiple usage modes based on transfer learning, which solves the problem that existing behavior recognition technology needs to collect more sensor data in new environments to calculate an accurate transformation matrix, reduces the workload of data collection, and speeds up data processing.

[0114] Exemplary devices

[0115] Based on the above embodiments, the present invention also provides a terminal, comprising: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein the processor is used to provide computing and control capabilities; the memory includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and computer program in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0116] When the computer program is executed by the processor, it is used to implement the operation of a behavior recognition method under multiple usage modes based on a generative adversarial network.

[0117] It will be understood by those skilled in the art that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In one embodiment, a computer terminal is provided, comprising: a processor and a memory, wherein the memory stores a behavior recognition program under multiple usage modes based on a generative adversarial network, and when the behavior recognition program under multiple usage modes based on a generative adversarial network is executed by the processor, it is used to implement the operation of the behavior recognition method under multiple usage modes based on a generative adversarial network as described above.

[0119] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a behavior recognition program under multiple usage modes based on a generative adversarial network, and when the behavior recognition program under multiple usage modes based on a generative adversarial network is executed by the processor, it is used to implement the operation of the behavior recognition method under multiple usage modes based on a generative adversarial network as described above.

[0120] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory.

[0121] In summary, the present invention provides a method and terminal for behavior recognition under multiple usage modes based on a generative adversarial network. The method includes: collecting behavior data under multiple usage modes to obtain a source domain dataset and a target domain dataset, and using the obtained source domain dataset and target domain dataset as inputs to a GAN model; extracting domain-invariant representations shared across domains between the target domain and the source domain through the GAN model, and generating a synthetic sequence in the target domain based on the label sequence of the source domain dataset; performing transfer learning based on the synthetic sequence generated from the target domain, and predicting behavior labels under multiple usage modes based on the learned domain-invariant representations. The present invention solves the problem that existing behavior recognition technologies require collecting more sensor data in new environments to calculate an accurate transformation matrix, reduces the workload of data collection, and speeds up data processing.

[0122] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A behavior recognition method under multiple usage modes based on generative adversarial networks, characterized in that: include: Collect behavioral data under multiple usage modes to obtain source domain datasets and target domain datasets, and use the obtained source domain datasets and target domain datasets as inputs to the GAN model; Extracting domain-invariant representations shared across domains between the target domain and the source domain through a GAN model, and generating synthetic sequences in the target domain based on the labeled sequence of the source domain dataset; generating synthetic sequences according to the target domain for transfer learning, and predicting behavioral labels under multiple usage modes based on the learned domain-invariant representations; The method of extracting domain-invariant representations shared across domains between the target domain and the source domain through the GAN model includes: The domain-invariant representation shared across domains between the target domain and the source domain is extracted through the RNN sequence encoder and the specific domain vector: Analyze common feature information extracted from sensor signals generated according to different behaviors in different usage modes, use different domain vectors in the source domain and the target domain to distinguish each domain-specific feature, and use preset function parameters to share the features between the source domain and the target domain; Generating a synthetic sequence in the target domain according to the labeled sequence of the source domain dataset includes: Combining the GAN generator with the RNN sequence encoder to obtain a sequence conversion model; generating a synthetic sequence in the target domain according to the sequence conversion model and the labeled sequence of the source domain dataset; The step of generating a synthetic sequence in the target domain according to the sequence conversion model and the labeled sequence of the source domain dataset further includes: According to cycle consistency regularization, the sequence generated by the source domain is mapped, and an improved generated synthetic sequence is obtained by defining a consistency loss penalty optimizer; According to perceptual consistency regularization, the RNN sequence encoder is used as a perceptual function to perform semantic representation constant after the GAN generator is converted to the target domain; The step of generating a synthetic sequence in the target domain according to the sequence conversion model and the labeled sequence of the source domain dataset further includes: Reconstructing the synthesized sequence by an RNN reconstruction decoder: reconstructing according to the domain invariant representation, the reconstruction loss function, and the noise pair obeying the normal distribution; The method of generating a synthetic sequence based on the target domain for transfer learning and predicting behavior labels under multiple usage modes based on the learned domain-invariant representation includes: Optimizing domain-invariant representations through the GAN generator and the RNN reconstruction decoder; When the extracted domain-invariant representation meets the requirements, the content learned in the source domain is transferred to the target domain through the behavior predictor shared by all domains; Predicting behavior labels in multiple usage modes based on the learned domain-invariant representation.

2. The method for behavior recognition under multiple usage modes based on generative adversarial networks according to claim 1 is characterized in that: The collecting of behavioral data in multiple usage modes to obtain a source domain dataset and a target domain dataset includes: Collecting behavioral data in handheld mode, using the behavioral data in handheld mode as a source domain dataset, and labeling corresponding behaviors in the source domain dataset; Behavioral data of other usage patterns are collected and used as target domain data.

3. The method for behavior recognition under multiple usage modes based on generative adversarial networks according to claim 1, characterized in that: The GAN model includes: the RNN sequence encoder, the GAN generator, the RNN reconstruction decoder, and the behavior predictor shared by all domains.

4. A computer terminal, characterized in that: include: A processor and a memory, wherein the memory stores a behavior recognition program under multiple usage modes based on a generative adversarial network, and when the behavior recognition program under multiple usage modes based on a generative adversarial network is executed by the processor, it is used to implement the operation of the behavior recognition method under multiple usage modes based on a generative adversarial network as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a behavior recognition program under multiple usage modes based on a generative adversarial network, and when the behavior recognition program under multiple usage modes based on a generative adversarial network is executed by a processor, it is used to implement the operation of the behavior recognition method under multiple usage modes based on a generative adversarial network as described in any one of claims 1 to 3.

Images