A method, device, medium and product for generating a motion signal in a Chinese context

Abstract

The application discloses a method and device for generating motion signals in a Chinese context, a medium and a product, and relates to the technical field of deep learning. The method comprises the following steps: acquiring an inertial sensor signal of a sample Chinese character; taking one-hot coding of the sample Chinese character as input, taking the inertial sensor signal of the sample Chinese character as output, training a forced optimal transport generative adversarial network to obtain a Chinese motion generative adversarial network; and inputting one-hot coding of a target Chinese character into the Chinese motion generative adversarial network to obtain the inertial sensor signal of the target Chinese character. The application can obtain a high-quality Chinese character handwriting dataset.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus, medium, and product for generating motion signals in a Chinese context. Background Technology

[0002] In the future information-driven and intelligent society, ubiquitous sensor systems will become the cornerstone of efficient social operation. Intelligent systems such as intelligent robots, smart homes, and autonomous driving will all rely on networks composed of various sensors. Therefore, to a certain extent, the accuracy of sensors will determine the level of intelligence in future human society. As one of the most representative sensors, MEMS inertial sensors are widely used in aerospace, autonomous driving, human-computer interaction, navigation and positioning, and other fields due to their advantages such as small size, easy wearability, low power consumption, low cost, and easy mass production. In fact, the unit price of common inertial sensors on the market can be as low as 0.2-0.5 yuan, thus possessing broad market prospects.

[0003] Character input, as the foundation of human-computer interaction, relies on the semantic recognition of text. Globally, approximately 1.4 billion people use Chinese characters, a logographic script that influences not only Chinese but also other Asian languages, including Japanese, Korean, and Vietnamese. Learning to write Chinese characters is a fundamental requirement in primary school education. Therefore, research into Handwritten Chinese Character Recognition (HCCR) technology has become a crucial task in the development of social informatization. Currently, people frequently need to transcribe written content from paper into computers. In this process, the application of handwritten Chinese character recognition technology is particularly important. Especially for beginners learning Chinese, they need auxiliary tools to help recognize and understand Chinese characters. The development of this technology not only helps improve the efficiency and accuracy of Chinese character input but also promotes the understanding and dissemination of Chinese culture. In this context, utilizing inertial sensors in smartphones to recognize handwritten Chinese characters is of great significance. Inertial sensors are small, low-power, and easily integrated into widely used smartphones. These sensors can accurately capture the motion and angular velocity during handwriting, making it possible to convert handwritten Chinese characters into digital text. Compared to traditional keyboard input, this method is more natural and conforms to handwriting habits, especially for a complex writing system like Chinese characters. Furthermore, inertial sensor-based Chinese character recognition technology not only provides a new avenue for the digitization of Chinese characters but also offers language learners a new learning tool. It helps users better understand the strokes and structure of Chinese characters, thereby improving learning efficiency. Simultaneously, the development of this technology can drive innovation in smartphone technology, making mobile phones even more powerful interactive tools. In conclusion, inertial sensor-based handwritten Chinese character recognition technology is not only significant for improving the convenience and accuracy of Chinese character input but also plays a crucial role in promoting the understanding and dissemination of Chinese characters and culture globally.

[0004] However, the lack of training samples makes it difficult to achieve Chinese handwriting recognition based on inertial sensors. As a complex language with a massive number of characters, it is impractical to collect a large number of handwriting samples for each character to train a deep learning classifier, as this would consume countless times the time and resources. Therefore, establishing a comprehensive, diverse, and high-quality Chinese handwriting dataset is a daunting task. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, medium, and product for generating motion signals in a Chinese context, which can obtain high-quality Chinese handwritten data sets.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for generating motion signals in a Chinese context, the method comprising:

[0008] Acquire inertial sensor signals from sample Chinese characters;

[0009] Using the one-hot encoding of the sample Chinese characters as input and the inertial sensor signal of the sample Chinese characters as output, a forced optimal transportation generative adversarial network is trained to obtain a Chinese motion generative adversarial network. The forced optimal transportation generative adversarial network incorporates a feature-forced matching mechanism, designs a pictographic encoding regularization term for the weight matrix of the fully connected layer, and adds a consistency constraint between the first feature similarity matrix and the second feature similarity matrix. The first feature similarity matrix is ​​the feature similarity matrix of the semantic encoding output by the fully connected layer; the second feature similarity matrix is ​​the feature similarity matrix of the inertial sensor generated signal output by the forced optimal transportation generative adversarial network.

[0010] The one-hot encoding of the target Chinese character is input into the Chinese motion generative adversarial network to obtain the inertial sensor signal of the target Chinese character.

[0011] Optionally, the training process of the forced optimal transport generative adversarial network includes:

[0012] The one-hot encoding of the sample Chinese characters is input into the forced optimal transport generative adversarial network. The consistency constraint between the first feature similarity matrix and the second feature similarity matrix is ​​applied. The loss function of the consistency constraint between the first feature similarity matrix and the second feature similarity matrix, the Wasserstein distance, and the feature forced matching loss are used as the total loss function to train the forced optimal transport generative adversarial network.

[0013] Optionally, the training process of the forced optimal transport generative adversarial network includes:

[0014] The one-hot encoding of the sample Chinese characters is input into the fully connected layer, and after being multiplied by the weight matrix of the fully connected layer with the application of the pictographic encoding regularization term, the semantic encoding is obtained; the forced optimal transport generative adversarial network includes a fully connected layer, a generative adversarial network and a pre-trained variational autoencoder connected in sequence.

[0015] The semantic encoding is input into the generative adversarial network, and the output is a signal generated by the inertial sensor;

[0016] Based on the consistency constraints between the signal generated by the inertial sensor and the semantic encoding, a first loss function is obtained;

[0017] A pre-trained variational autoencoder is applied to map the inertial sensor generated signal and the inertial sensor signal of the sample Chinese character into the feature space;

[0018] Based on the consistency constraint between the inertial sensor generated signal and the inertial sensor signal of the sample Chinese character in the feature space, a second loss function is obtained;

[0019] The forced optimal transport generative adversarial network is trained using the first loss function, the second loss function, and the Wasserstein distance as the total loss function.

[0020] Optionally, the pictographic encoding regularization term is:

[0021]

[0022]

[0023]

[0024] G[i][j]= <W (i) W (j) >;

[0025] Where N represents the number of Chinese characters, which is the α-order Rényi entropy measure of the number of rows in the pictographic encoding matrix W; G is the Gram matrix of the encoding matrix W; G[i][j] is the element in the i-th row and j-th column of G; Normalize the trace of G[i][j]; S α (W) represents the information content of the glyph encoding matrix W; W is the α-order Rényi entropy pictographic encoding matrix; W (i) The i-th row of the encoding matrix W; W (j) The j-th row of the encoding matrix W; <·> represents the inner product; R encoding For pictographic encoding regular terms; A mathematical formula refers to a matrix. The traces.

[0026] Optionally, the first loss function is:

[0027] L bridge =||M f -M e || 2 ;

[0028] Among them, ||·|| 2 M represents the sum of squares of the matrix elements; f Generate a feature similarity matrix for the signal generated by the inertial sensor; M e The feature similarity matrix for semantic encoding; L bridge This is the first loss function.

[0029] Optionally, the second loss function is:

[0030]

[0031]

[0032] in, and Let x and y represent the data distributions of the real signal and the generated signal, respectively. Let represent the real sample and the generated sample respectively, d represent the cosine distance in the feature space; F(x) is the vector of the real signal in the feature space; To generate a vector of the signal in the feature space, θ is F(x) and The angle between them, ||F(x)|| and F(x) and Length, F(x) and The inner product of.

[0033] Optionally, the Chinese motion generative adversarial network includes a fully connected layer and a generative adversarial network connected in sequence.

[0034] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the method for generating motion signals in a Chinese context as described in any of the preceding claims.

[0035] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating motion signals in a Chinese context as described in any of the preceding claims.

[0036] A computer program product includes a computer program that, when executed by a processor, implements the method for generating motion signals in a Chinese context as described in any of the preceding claims.

[0037] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0038] The Forced Optimal Transport Generative Adversarial Network (GAN) provided in this invention designs a feature-forced matching mechanism based on GANs and introduces optimal transport theory. It utilizes Wasserstein distance to solve the pattern collapse and aliasing problems of traditional GAN ​​architectures, ensuring feature consistency between the generated new signal and the real signal. Furthermore, it designs a pictographic encoding regularization term for the weight matrix of the fully connected layer, which can improve the orthogonality of all Chinese character encoding vectors and increase the information entropy of all Chinese character encoding vectors to store as many glyph characteristics of Chinese characters as possible. In addition, it adds consistency constraints between the first and second feature similarity matrices. The first feature similarity matrix is ​​the feature similarity matrix of the semantic encoding output of the fully connected layer; the second feature similarity matrix is ​​the feature similarity matrix of the inertial sensor generated signal output by the Forced Optimal Transport GAN. By minimizing the difference between the feature similarity matrix and the encoding similarity matrix through the consistency constraints of the first and second feature similarity matrices, the generative model is guided to focus on the correlation of different Chinese character writing movements, enabling the generated inertial sensor signal to more accurately reflect the Chinese character writing process. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a diagram of the Chinese motion generative adversarial network framework of this invention;

[0041] Figure 2 This is a schematic diagram of the semantic connectivity bridge constraint of the present invention;

[0042] Figure 3 This is a schematic diagram of the method for generating motion signals in a Chinese context according to Embodiment 1 of the present invention;

[0043] Figure 4 This is a diagram of the internal structure of a computer device. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] The purpose of this invention is to provide a method, apparatus, medium, and product for generating motion signals in a Chinese context, with the aim of obtaining a high-quality Chinese handwritten data set.

[0046] This invention proposes a method for automatically generating motion signals of Chinese character writing: Chinese Moving Generative Adversarial Network (CM-GAN).

[0047] Currently, there are no methods for generating writing signals for inertial sensors. This invention is the first to realize the generation of simulated motion signals based on input semantics, which plays a crucial role in the transformation of large language models into intelligent robots. Furthermore, it is the first to propose encoding the semantic characteristics of Chinese characters, such as their morphology and strokes during writing. This not only injects action semantics into the generation process of Chinese character writing signals but also provides new tools and perspectives for the evolution and understanding of pictographic characters.

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] like Figure 3 As shown, the method for generating motion signals in a Chinese context in this embodiment includes:

[0051] Step S1: Obtain the inertial sensor signal of the sample Chinese character.

[0052] Step S2: Using the one-hot encoding of the sample Chinese characters as input and the inertial sensor signal of the sample Chinese characters as output, train the Forced Optimal Transportation Generative Adversarial Network (GAN) to obtain a Chinese motion GAN. The Forced Optimal Transportation GAN is designed with a feature forced matching mechanism based on the GAN, and a pictographic encoding regularization term is designed for the weight matrix of the fully connected layer. Furthermore, a consistency constraint between the first feature similarity matrix and the second feature similarity matrix is ​​added. The first feature similarity matrix is ​​the feature similarity matrix of the semantic encoding output by the fully connected layer. The second feature similarity matrix is ​​the feature similarity matrix of the inertial sensor generated signal output by the Forced Optimal Transportation GAN.

[0053] Step S3: Input the one-hot encoding of the target Chinese character into the Chinese motion generative adversarial network to obtain the inertial sensor signal of the target Chinese character.

[0054] As a specific implementation, the training process of the forced optimal transport generative adversarial network includes:

[0055] The one-hot encoding of the sample Chinese characters is input into the forced optimal transport generative adversarial network. The consistency constraint between the first feature similarity matrix and the second feature similarity matrix is ​​applied. The loss function of the consistency constraint between the first feature similarity matrix and the second feature similarity matrix, the Wasserstein distance, and the feature forced matching loss are used as the total loss function to train the forced optimal transport generative adversarial network.

[0056] As a specific implementation, the training process of the forced optimal transport generative adversarial network includes:

[0057] Step 101: The one-hot encoding of the sample Chinese character is input into the fully connected layer, and multiplied with the weight matrix of the fully connected layer with the applied pictographic encoding regularization term to obtain the semantic encoding; the forced optimal transport generative adversarial network includes a fully connected layer, a generative adversarial network and a pre-trained variational autoencoder connected in sequence.

[0058] Step 102: The semantic encoding is input into the generative adversarial network, and the output is the signal generated by the inertial sensor.

[0059] Step 103: Obtain the first loss function based on the consistency constraint between the signal generated by the inertial sensor and the semantic encoding.

[0060] Step 104: Apply a pre-trained variational autoencoder to map the inertial sensor generated signal and the inertial sensor signal of the sample Chinese character into the feature space.

[0061] Step 105: Based on the consistency constraint between the inertial sensor generated signal and the inertial sensor signal of the sample Chinese character in the feature space, obtain the second loss function.

[0062] Step 106: Train the forced optimal transport generative adversarial network using the first loss function, the second loss function, and the Wasserstein distance as the total loss function.

[0063] The Chinese motion generative adversarial network includes a fully connected layer and an adversarial generative network after the forced optimal transport generative adversarial network has been trained, and the fully connected layer and the adversarial generative network of the Chinese motion generative adversarial network are connected sequentially.

[0064] In practical applications, to generate the handwriting inertial signal of each Chinese character, the present invention designs a Chinese motion generative adversarial network, the structure of which is shown in Figure 1 . For an input Chinese character, its one-hot encoding is converted into a semantic encoding by the Chinese encoding dictionary designed by the present invention, and the semantic encoding is then input into a conditional GAN, so as to generate the writing IMU signal of the Chinese character. To ensure that the generative model can learn the IMU signal pattern of each Chinese character, the present invention designs a "forced optimal transport loss" for the GAN, which makes the generated IMU signal closer to the real handwriting signal. The details are as follows:

[0065] 1. Chinese character glyph encoding.

[0066] In existing studies, conditions are often input into the conditional generative adversarial network in the form of one-hot encoding. However, under this encoding method, each Chinese character is represented by a high-dimensional sparse vector (in which only one element is 1 and the rest are 0), which will cause all Chinese characters to be equidistant from each other in the vector space, thus losing the complex semantic correlation between Chinese characters. Therefore, one-hot encoding cannot inject accurate semantic information into the CGAN. Although there are some commonly used Chinese character embedding representations at present, these embedding vectors store the meaning information of characters rather than the glyph information (that is, the motion semantics during writing). For example, the Chinese characters "tian" (meaning sky) and "fu" (meaning husband) have extremely similar motions when written, but their meanings are quite different. Therefore, the present invention designs a "Chinese Glyph Encoding (CGE)" method, which encodes and represents Chinese characters according to their glyphs and writing motions.

[0067] Since the collected inertial sensor signals store the writing information of Chinese characters, and this writing information implies the glyph information of Chinese characters, the present invention can encode such glyph information in a learnable weight matrix through backpropagation of loss in the entire CI-GAN signal generation framework. To this end, after a Chinese character is input into the CI-GAN in the form of one-hot encoding, the present invention first passes it through a fully connected layer. Since only one element in the one-hot encoding is 1 and the rest are 0, multiplying by the weight matrix $W$ of the fully connected layer means obtaining one row of the matrix. Therefore, each row of the weight matrix of the fully connected layer can be regarded as the encoding of a Chinese character, and the matrix can undertake the task of encoding the glyph of the Chinese character, so it is called: Glyph Encoding Dictionary.

[0068] However, unconstrained Chinese encoding dictionaries often struggle to capture the differences in the shapes of different Chinese characters, thus assigning very similar pictographic codes to characters with vastly different shapes. To force Chinese encoding dictionaries to capture these differences as much as possible, this invention designs a Glyph Encoding Regularization (GER) term, which improves the orthogonality of all Chinese character encoding vectors. Simultaneously, it increases the information entropy of all Chinese character encoding vectors to store as many glyph characteristics as possible, thus avoiding the simplistic and trivial nature of one-hot encoding.

[0069] Specifically, this invention first uses the α-order Rényi entropy to measure the information content of the pictographic coding dictionary W. The α-order Rényi entropy to measure the pictographic coding dictionary W is the α-order Rényi entropy to measure the pictographic coding matrix W in this invention. The calculation process is as follows:

[0070]

[0071]

[0072] G[i][j]= <W (i) W (j) >;

[0073] Where N represents the number of Chinese characters, i.e., the number of rows in the pictographic encoding matrix W, which is the Rényi entropy of order α. G represents the Gram matrix of the encoding matrix W, where the element G[i][j] in the i-th row and j-th column of G is equal to the inner product of the i-th and j-th rows of matrix W. For the trace normalization of G[i][j], i.e. In similar problems, taking α to be 2 generally yields better results. α (W) measured the information content of the glyph encoding matrix W, S α The larger (W) is, the more information is encoded in W, which means that these Chinese character glyph codes are more information-rich. At the same time, with S... α As (W) increases, all elements in the Gram matrix G are forced to decrease, meaning the inner product of different encoded vectors is smaller, i.e., different codes have stronger orthogonality. It can be observed that S... α Increasing the value of (W) can simultaneously increase the information content of all codes and the orthogonality between codes. Therefore, this invention proposes a regularization term R. encoding as follows:

[0074]

[0075] As the regularization term decreases during the training process, the present invention can obtain the accurate glyph encoding of each Chinese character. This encoding can not only better control the direction of the generated signal, but also inject sufficient glyph semantic information into the generation process.

[0076] 2. Semantic Bridges Constraint.

[0077] The core idea of the Semantic Bridges Constraint is to construct a bridge between the input and output of the generative model, and utilize the consistency of the relative relationships between multiple inputs of the generative model and the relative relationships between the corresponding outputs, to force the generative model to capture and understand the deep semantic relationships of the data, so that it generates more realistic and accurate samples, and its overall architecture is as shown in Figure 2 the figure below.

[0078] Specifically, assuming that the present invention inputs the encodings X = [x1, x2, ..., xn] of the eight characters "fu", "tian", "tai", "da", "quan", "ren", "ba", "ru" to the generator, the generator will output the inertial sensor signals Y = [y1, y2, ..., yn] corresponding to the writing of these eight characters. Considering that different Chinese characters have semantic correlations such as morphological similarity and stroke similarity, such correlations should be reflected in both the input semantic encodings and the generated sensor signals. In simple terms, characters with similar strokes should have similar semantic encodings and similar writing motion signals. To this end, the present invention uses a pre-trained Autoencoder (AE) to convert each generated signal y i into a 1×100 feature vector z i , and these feature vectors are used to calculate a feature similarity matrix M f , where the matrix element M f [i][j] represents the cosine similarity between the feature vector z i and z j . Similarly, the present invention also calculates the cosine similarity between the Chinese character encodings to obtain an encoding similarity matrix M e . Both of these matrices contain action semantic correlations such as stroke similarity of Chinese characters, so the two should be highly consistent in theory. Therefore, the present invention defines the Semantic Bridges Constraint as follows:

[0079] L bridge =||M f -M e || 2 ;

[0080] where ||·|| 2The loss function, which represents the sum of squares of the matrix elements, guides the generative model to focus on the correlation between different Chinese character writing movements by minimizing the difference between the feature similarity matrix and the encoding similarity matrix. This makes the generated IMU signal more accurately reflect the writing process of Chinese characters.

[0081] 3. Forced optimal transport generative adversarial networks.

[0082] Traditional GANs often suffer from pattern collapse and aliasing when generating complex data (such as inertial sensor signals simulating handwriting). Pattern collapse refers to the generator learning only a small number of signal patterns, resulting in highly similar outputs under diverse input conditions. In the context of this invention, this means that even with different Chinese character inputs, the generator can only generate sensor simulation signals for certain specific characters, which clearly fails to meet the requirement of signal diversity in this invention. On the other hand, aliasing refers to the generator output signal falling between two or more patterns, resulting in a "hybrid" signal that does not belong to any specific pattern. For example, when generating inertial sensor signals simulating Chinese character writing, signals exhibiting aliasing may possess the writing characteristics of multiple Chinese characters, a phenomenon that obviously does not exist in practical applications.

[0083] To address these issues, this invention introduces Optimal Transport (OT) theory on top of GANs for the first time, and utilizes Wasserstein distance to solve the mode collapse and aliasing problems of traditional GAN ​​architectures. Furthermore, it is the first to apply a GAN incorporating OT theory to inertial sensor signal generation. Traditional GANs use JS divergence as the loss function. When two distributions differ significantly or even do not overlap, the JS divergence becomes zero, causing the model to fail to optimize further, thus triggering mode collapse. OT-GAN uses Wasserstein distance as the loss function, which still provides meaningful gradient information even when two distributions do not overlap or are far apart, thereby avoiding mode collapse. Moreover, the JS loss in traditional GANs, when the capabilities of the generator and discriminator are mismatched, can induce the generator to tend to generate data that can deceive the discriminator instead of real data, leading to aliasing. Wasserstein distance, however, is insensitive to the training balance of the generator and discriminator, thus avoiding aliasing.

[0084] While the introduction of OT loss improves sample diversity across the overall distribution, the realism of individual signal samples still needs improvement. Specifically, generated signals differ significantly from real signals in many features, such as acceleration variations in specific strokes or angular velocity changes at turning points. To address this, this invention proposes a Forced Optimal Transport Generative Adversarial Network (FOT-GAN), which, based on OT-GAN, adds a feature-forced matching mechanism designed in this invention to ensure feature consistency between generated and real signals. Specifically, this invention uses a pre-trained variational autoencoder F to map generated and real signals to a feature space, and then evaluates their consistency C in the feature space. The calculation process is as follows:

[0085]

[0086] in, and Let x and y represent the data distributions of the real signal and the generated signal, respectively. Let represent the real sample and the generated sample respectively, and d represent the cosine distance in the feature space, as expressed below:

[0087]

[0088] Where A and B are two vectors, θ is the angle between them, ||A|| and ||B|| are their lengths, and A·B is their inner product.

[0089] In this invention,

[0090] in, and Let x and y represent the data distributions of the real signal and the generated signal, respectively. Let represent the real sample and the generated sample respectively, d represent the cosine distance in the feature space; F(x) is the vector of the real signal in the feature space; To generate a vector of the signal in the feature space, θ is F(x) and The angle between them, ||F(x)|| and F(x) and Length, F(x) and The inner product of.

[0091] This invention adds C as a forced feature matching loss to the total loss function L:

[0092]

[0093] Here, W represents the Wasserstein distance, used to optimize the overall distribution of generated samples to improve their diversity and stability. λ is the weighting coefficient of the forced feature matching loss. As C decreases during training, the generated signal will gradually approach the real signal at the feature level.

[0094] The present invention has the following technical effects:

[0095] 1. This invention is the first to realize the generation of Chinese character writing signals based on inertial sensors. This technology not only provides a new way for the digitization of Chinese characters, but also provides a new learning tool for language learners, playing a key role in promoting the understanding and dissemination of Chinese characters and Chinese culture worldwide.

[0096] 2. This invention designs a Chinese Moving Generative Adversarial Network (CM-GAN), which can generate corresponding inertial motion signals based on input Chinese characters. This technology of generating motion signals from Chinese semantics (characters) is a key step in the transition from "language model" to "intelligent robot". Furthermore, CI-GAN can generate an unlimited number of signal samples, thus providing a flexible and convenient experimental data platform for human-computer interaction research based on Chinese handwriting recognition.

[0097] 3. This invention proposes a pictographic encoding method for Chinese characters. Compared with traditional Chinese character vectorization methods that only consider the semantic correlation between different characters, CGE mines the semantic information contained in the shape and writing process of Chinese characters. It can not only inject action semantics into the generation process of Chinese character writing signals, but also provide new tools and perspectives for the evolution and understanding of pictographic characters.

[0098] 4. This invention proposes a semantic connectivity bridge constraint, which utilizes the consistency between the relative relationships between multiple inputs of the generative model and the relative relationships between their corresponding outputs to force the generative model to capture and understand the deep semantic relationships of the data, thereby enabling it to generate more realistic and accurate samples.

[0099] 5. This invention proposes a Forced Optimal Transport Generative Adversarial Network (FOT-GAN), which not only avoids mode collapse and aliasing during signal generation, but also ensures the consistency of features between the generated signal and the real signal through a designed forced feature matching mechanism, thereby improving the authenticity of the generated signal.

[0100] Example 2

[0101] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the method for generating motion signals in a Chinese context as described in Embodiment 1.

[0102] Example 3

[0103] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating motion signals in a Chinese context as described in Embodiment 1.

[0104] Example 4

[0105] A computer program product includes a computer program that, when executed by a processor, implements the method for generating motion signals in the Chinese context described in Embodiment 1.

[0106] Example 5

[0107] A computer device, which may be a database, may have an internal structure diagram as shown below. Figure 4 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores pending transactions. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the motion signal generation method in the Chinese context described in Embodiment 1.

[0108] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0109] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for generating motion signals in a Chinese context, characterized in that, The method includes: Acquire inertial sensor signals from sample Chinese characters; Using the one-hot encoding of the sample Chinese characters as input and the inertial sensor signal of the sample Chinese characters as output, a forced optimal transportation generative adversarial network is trained to obtain a Chinese motion generative adversarial network. The forced optimal transportation generative adversarial network incorporates a feature-forced matching mechanism, designs a pictographic encoding regularization term for the weight matrix of the fully connected layer, and adds a consistency constraint between the first feature similarity matrix and the second feature similarity matrix. The first feature similarity matrix is ​​the feature similarity matrix of the semantic encoding output by the fully connected layer; the second feature similarity matrix is ​​the feature similarity matrix of the inertial sensor generated signal output by the forced optimal transportation generative adversarial network. The training process of the forced optimal transport generative adversarial network includes: inputting the one-hot encoding of the sample Chinese character into the fully connected layer, multiplying it with the weight matrix of the fully connected layer after applying a pictographic encoding regularization term to obtain a semantic encoding; the forced optimal transport generative adversarial network includes a fully connected layer, a generative adversarial network, and a pre-trained variational autoencoder connected in sequence; inputting the semantic encoding into the generative adversarial network, outputting an inertial sensor generated signal; obtaining a first loss function based on the consistency constraint between the inertial sensor generated signal and the semantic encoding; applying the pre-trained variational autoencoder to map the inertial sensor generated signal and the inertial sensor signal of the sample Chinese character into a feature space; obtaining a second loss function based on the consistency constraint between the inertial sensor generated signal and the inertial sensor signal of the sample Chinese character in the feature space; and training the forced optimal transport generative adversarial network using the first loss function, the second loss function, and the Wasserstein distance as the total loss function. The pictographic encoding regular expression is: ； ； ； ； Where N represents the number of Chinese characters, that is, the number of rows in the pictographic encoding matrix W, which is the Rényi entropy of order α. is the Gram matrix of the pictographic encoding matrix W; for The element in the i-th row and j-th column; for Trace normalization; ; W represents the information content of the pictographic encoding matrix W; W is the α-order Rényi entropy metric pictographic encoding matrix; W (i) The i-th row of the pictographic coding matrix W; W (j) The j-th row of the pictographic coding matrix W; It is the inner product; For pictographic encoding regular terms; A mathematical formula refers to a matrix. traces; The one-hot encoding of the target Chinese character is input into the Chinese motion generative adversarial network to obtain the inertial sensor signal of the target Chinese character.

2. The method for generating motion signals in a Chinese context according to claim 1, characterized in that, The training process of the forced optimal transport generative adversarial network includes: The one-hot encoding of the sample Chinese characters is input into the forced optimal transport generative adversarial network. The consistency constraint between the first feature similarity matrix and the second feature similarity matrix is ​​applied. The loss function of the consistency constraint between the first feature similarity matrix and the second feature similarity matrix, the Wasserstein distance, and the feature forced matching loss are used as the total loss function to train the forced optimal transport generative adversarial network.

3. The method for generating motion signals in a Chinese context according to claim 1, characterized in that, The first loss function is: ； in, M represents the sum of squares of the matrix elements; f Generate a feature similarity matrix for the signal from the inertial sensor; The feature similarity matrix is ​​used for semantic encoding; This is the first loss function.

4. The method for generating motion signals in a Chinese context according to claim 1, characterized in that, The second loss function is: ； ； in, and Let x and y represent the data distributions of the real signal and the generated signal, respectively. Let represent real samples and generated samples respectively, and d represent the cosine distance in the feature space; This is the vector of the real signal in the feature space; To generate a vector of the signal in the feature space, θ is and The angle between them, || ||and|| || are respectively and Length, yes and The inner product of.

5. The method for generating motion signals in a Chinese context according to claim 1, characterized in that, The Chinese motion generative adversarial network comprises a fully connected layer and an adversarial generative network connected in sequence.

6. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for generating motion signals in the Chinese context as described in any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for generating motion signals in the Chinese context as described in any one of claims 1-5.

8. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method for generating motion signals in the Chinese context as described in any one of claims 1-5.

Images