Vestibular illusion training type verification method

Abstract

The invention relates to the technical field of vestibular illusion training, in particular to a vestibular illusion training type verification method. Obtaining a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type; performing vestibular illusion training type verification on a preset person based on the training data, and obtaining motion parameters corresponding to the preset person; performing data processing on the motion parameters to generate target data; based on a preset type identification model, identifying the target data, and determining a target vestibular illusion identification type corresponding to the target data; comparing the target vestibular illusion recognition type with a preset vestibular illusion training type; and verifying the training data according to a comparison result. Therefore, the accuracy of the training data corresponding to the vestibular illusion training type can be ensured, and the verification accuracy of the vestibular illusion training type can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of vestibular illusion training, and in particular to a vestibular illusion training type verification method. Background Art

[0002] Spatial disorientation is one of the problems that almost all pilots have encountered during flight. Among them, vestibular illusion is caused by the physiological limitations of the human vestibular system and is also one of the types of spatial disorientation that pilots find difficult to deal with.

[0003] Existing vestibular illusion training usually generates training data corresponding to each vestibular illusion training type based on historical training data or historical experience. However, the generated training data corresponding to the vestibular illusion training type is not accurate, which may lead to inaccurate results when performing different vestibular illusion type training on preset personnel.

[0004] Therefore, how to improve the accuracy of the training data corresponding to the vestibular illusion training type, and then improve the accuracy of the vestibular illusion type training, has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present invention provides a vestibular illusion training type verification method to solve the problem of how to improve the accuracy of training data corresponding to the vestibular illusion training type, thereby improving the accuracy of vestibular illusion type training.

[0006] In a first aspect, the present invention provides a method for verifying a vestibular illusion training type, the method comprising: Obtaining a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type; Perform vestibular illusion type training on a preset person based on the training data, and obtain motion parameters corresponding to the preset person; Perform data processing on motion parameters to generate target data; Based on a preset type recognition model, the target data is recognized to determine the target vestibular illusion recognition type corresponding to the target data; The target vestibular illusion recognition type was compared with the preset vestibular illusion training type; According to the comparison results, the training data is verified.

[0007] The vestibular illusion training type verification method provided in the embodiment of the present application obtains a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type; performs vestibular illusion type training on a preset person based on the training data, and obtains motion parameters corresponding to the preset person, thereby ensuring the accuracy of the acquired motion parameters. Data processing is performed on the motion parameters to generate target data, thereby ensuring the accuracy of the generated target data. Based on a preset type recognition model, the target data is identified, and the target vestibular illusion recognition type corresponding to the target data is determined, thereby ensuring the accuracy of the target vestibular illusion recognition type corresponding to the determined target data. The target vestibular illusion recognition type is compared with the preset vestibular illusion training type to ensure the accuracy of the generated comparison result. According to the comparison result, the training data is verified to ensure the accuracy of the verification of the training data. Thus, the accuracy of the training data corresponding to the vestibular illusion training type can be guaranteed, thereby improving the accuracy of the vestibular illusion type training.

[0008] In an optional implementation manner, the target data is a target color image; and performing data processing on the motion parameters to generate the target data includes: Performing image conversion processing on the motion parameters to generate a grayscale image corresponding to the motion parameters; Generate a target color image based on the grayscale image.

[0009] The vestibular illusion training type verification method provided in the embodiment of the present application performs image conversion processing on motion parameters to generate grayscale images corresponding to the motion parameters, thereby ensuring the accuracy of the grayscale images corresponding to the generated motion parameters. Based on the grayscale image, a target color image is generated to ensure the accuracy of the generated target color image, thereby ensuring the accuracy of the target vestibular illusion recognition type corresponding to the target color image determined by identifying the target color image based on a preset type recognition model.

[0010] In an optional implementation, the motion parameters include a head motion parameter sequence and a body motion parameter sequence, and the grayscale image includes a head grayscale image and a body grayscale image; performing image conversion processing on the motion parameters to generate a grayscale image corresponding to the motion parameters includes: Perform data conversion on the head motion parameter sequence to generate a head grayscale image; Perform data conversion on the body motion parameter sequence to generate a body grayscale image.

[0011] The vestibular illusion training type verification method provided in the embodiment of the present application performs data conversion on the head motion parameter sequence to generate a head grayscale image, thereby ensuring the accuracy of the generated head grayscale image. Performs data conversion on the body motion parameter sequence to generate a body grayscale image, thereby ensuring the accuracy of the generated body grayscale image.

[0012] In an optional implementation, performing data conversion on the head motion parameter sequence to generate a head grayscale image includes: Splitting the head motion parameters to generate multiple non-intersecting head motion parameter subsequences; Calculating the correlation integral between the head motion parameter subsequences according to the relationship between the head motion parameter subsequences; Calculating the autocorrelation characteristics between the head motion parameter subsequences according to the correlation integral between the head motion parameter subsequences; Determine the delay time and embedding dimension corresponding to the head motion parameter sequence according to the autocorrelation characteristics between each head motion parameter subsequence; Based on the delay time and embedding dimension, the head phase space corresponding to the head motion parameter sequence is generated; A head grayscale image is generated according to the distance between each head vector in the head phase space.

[0013] The vestibular illusion training type verification method provided by the embodiment of the present application ensures the accuracy of the correlation integral and autocorrelation characteristics between the calculated head motion parameter subsequences. According to the autocorrelation characteristics between each head motion parameter subsequence, the delay time and embedding dimension corresponding to the head motion parameter sequence are determined. Based on the delay time and embedding dimension, the head phase space corresponding to the head motion parameter sequence is generated. According to the distance between each head vector in the head phase space, a head grayscale image is generated, which ensures the accuracy of the generated head grayscale image.

[0014] In an optional embodiment, the head motion parameter sequence includes a linear acceleration motion parameter sequence of the head in the X, Y, and Z axes and an angular acceleration motion parameter sequence of the head in the X, Y, and Z axes; the head grayscale image includes an acceleration grayscale image of the head in the X-axis line, an acceleration grayscale image of the head in the Y-axis line, an acceleration grayscale image of the head in the Z-axis line, an angular acceleration grayscale image of the head in the X-axis, an angular acceleration grayscale image of the head in the Y-axis, and an angular acceleration grayscale image of the head in the Z-axis.

[0015] In an optional implementation, the grayscale image includes a head grayscale image and a body grayscale image, and generating a target color image based on the grayscale image includes: Generate a color image of the head based on the grayscale image of the head; Generate a body color image based on the body grayscale image; The head color image and the body color image are spliced ​​to generate a target color image.

[0016] The vestibular illusion training type verification method provided in the embodiment of the present application generates a head color image based on the head grayscale image, thereby ensuring the accuracy of the generated head color image. Generates a body color image based on the body grayscale image, thereby ensuring the accuracy of the generated body color image. The head color image and the body color image are spliced ​​to generate a target color image, thereby ensuring the accuracy of the generated target color image.

[0017] In an optional embodiment, the head grayscale image includes the head acceleration grayscale image on the X axis, the head acceleration grayscale image on the Y axis, the head acceleration grayscale image on the Z axis, the head angular acceleration grayscale image on the X axis, the head angular acceleration grayscale image on the Y axis, and the head angular acceleration grayscale image on the Z axis, and the head color image includes the head linear acceleration color image and the head angular acceleration color image; based on the head grayscale image, generating the head color image includes: Determine a first red channel from the grayscale image of the acceleration of the head on the X axis, the grayscale image of the acceleration of the head on the Y axis, and the grayscale image of the acceleration of the head on the Z axis, determine a first green channel from the two grayscale images except the first red channel, and determine the last grayscale image as the first blue channel; Generate a head linear acceleration color image according to the determined first red channel, first green channel and first blue channel; Determine a second red channel from the grayscale image of the angular acceleration of the head on the X-axis, the grayscale image of the angular acceleration of the head on the Y-axis, and the grayscale image of the angular acceleration of the head on the Z-axis, determine a second green channel from the two grayscale images except the second red channel, and determine the last grayscale image as the second blue channel; A head angular acceleration color image is generated according to the determined second red channel, second green channel, and second blue channel.

[0018] The vestibular illusion training type verification method provided in the embodiment of the present application ensures that the generated head linear acceleration color image can represent the linear acceleration motion parameter sequence of the head in the X, Y, and Z axes. It ensures that the generated head angular acceleration color image can represent the angular acceleration motion parameter sequence of the head in the X, Y, and Z axes.

[0019] In an optional implementation, the target data is identified based on a preset type recognition model to determine the target vestibular illusion recognition type corresponding to the target data, including: Inputting the target color image into a preset type recognition model; The preset type recognition model performs feature recognition and feature extraction on the target color image, and outputs the target vestibular illusion recognition type based on the extracted features.

[0020] The vestibular illusion training type verification method provided in the embodiment of the present application inputs the target color image into a preset type recognition model; the preset type recognition model performs feature recognition and feature extraction on the target color image, and outputs the target vestibular illusion recognition type based on the extracted features, thereby ensuring the accuracy of the output target vestibular illusion recognition type.

[0021] In an optional implementation, the preset type recognition model includes a partitioning layer, a Swin Transformer layer, a pooling layer, a fully connected layer, and a Softmax layer. The preset type recognition model performs feature recognition and feature extraction on the target color image, and outputs the target vestibular illusion recognition type based on the extracted features, including: The division layer divides the target color image to generate a plurality of target color image blocks, and generates label information corresponding to each target color image block; The Swin Transformer layer fuses the feature information corresponding to each target color image block to generate a fused feature, and performs a linear transformation on the channel data of each pixel in the fused feature to generate a multi-dimensional feature; it extracts features from the multi-dimensional feature to generate the target feature; The pooling layer compresses and extracts the target features to generate local features; The fully connected layer integrates local features to generate global features and maps the global features to output categories; The Softmax layer converts the output category of the fully connected layer output into a probability distribution, and determines the target vestibular illusion recognition type based on the probability distribution.

[0022] The vestibular illusion training type verification method provided in the embodiment of the present application is as follows: the partitioning layer partitions the target color image to generate multiple target color image blocks, and generates labeling information corresponding to each target color image block; the SwinTransformer layer fuses the feature information corresponding to each target color image block to generate a fused feature, and performs a linear transformation on the channel data of each pixel in the fused feature to generate a multi-dimensional feature; the multi-dimensional feature is subjected to feature extraction to generate a target feature, thereby ensuring the accuracy of the generated target feature. The pooling layer compresses the target feature and extracts it to generate a local feature, thereby ensuring the accuracy of the generated local feature. The fully connected layer integrates the local features to generate a global feature, and maps the global feature to the output category; the Softmax layer converts the output category of the fully connected layer output into a probability distribution, and determines the target vestibular illusion recognition type based on the probability distribution, thereby ensuring the accuracy of the determined target vestibular illusion recognition type.

[0023] In an optional implementation, the Swin Transformer layer includes four stages, the first stage includes a first normalization layer and a window multi-head self-attention layer, the second stage includes a second normalization layer and a first multi-layer perceptron layer, the third stage includes a third normalization layer and a multi-head self-attention layer based on a shifted window, and the fourth stage includes a fourth normalization layer and a second multi-layer perceptron layer; the Swin Transformer layer fuses feature information corresponding to each target color image block to generate a fused feature, and performs a linear transformation on the channel data of each pixel in the fused feature to generate a multi-dimensional feature; and extracts features from the multi-dimensional feature to generate a target feature, including: The first normalization layer normalizes the fused features to generate the first normalized features; The window multi-head self-attention layer performs self-attention calculation on the first normalized feature and performs residual calculation to generate the first attention feature; The second normalization layer normalizes the first attention feature to generate a second normalized feature; The first multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the second normalized feature to generate a multi-dimensional feature; The third normalization layer normalizes the multi-dimensional features to generate the third normalized features; The multi-head self-attention layer based on the shifted window performs self-attention calculation on the third normalized feature to generate the second attention feature; The fourth normalization layer normalizes the second attention feature to generate a fourth normalized feature; The second multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the fourth normalized feature to generate the target feature.

[0024] The vestibular illusion training type verification method provided by the embodiment of the present application, the first normalization layer normalizes the fusion feature to generate the first normalized feature, so that the preset type recognition model has stronger robustness to the scale change of the input. The window multi-head self-attention layer performs self-attention calculation on the first normalized feature, and performs residual calculation to generate the first attention feature, so that the multi-head mechanism can be used to simultaneously focus on multiple different representation subspaces, thereby improving the learning ability and generalization performance of the preset type recognition model. The second normalization layer normalizes the first attention feature to generate the second normalized feature; the first multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the second normalized feature to generate a multi-dimensional feature, which increases the expression ability of the preset type recognition model. The third normalization layer normalizes the multi-dimensional feature to generate a third normalized feature; the multi-head self-attention layer based on the shifted window performs self-attention calculation on the third normalized feature to generate the second attention feature, which helps to better model long-distance dependencies and further improve the feature representation ability of the preset type recognition model. The fourth normalization layer normalizes the second attention feature to generate a fourth normalized feature; the second multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the fourth normalized feature to generate a target feature, thereby ensuring the accuracy of the generated target feature. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 is a flow chart of a method for verifying vestibular illusion training type according to an embodiment of the present invention; Figure 2 is a flow chart of another vestibular illusion training type verification method according to an embodiment of the present invention; Figure 3 is a schematic diagram of a process of converting motion parameters to generate a target color image according to an embodiment of the present invention; Figure 4 is a flow chart of another vestibular illusion training type verification method according to an embodiment of the present invention; Figure 5 is a schematic diagram of a flow chart of identifying a target color image and determining a target vestibular illusion recognition type according to an embodiment of the present invention; Figure 6is a schematic diagram of a Swin Transformer network structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] It should be noted that the method for verifying the vestibular illusion training type provided in the embodiment of the present application, its execution subject can be a device for verifying the vestibular illusion training type, and the device for verifying the vestibular illusion training type can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware, wherein the electronic device can be a server or a terminal, wherein the server in the embodiment of the present application can be a single server or a server cluster composed of multiple servers, and the terminal in the embodiment of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, and other intelligent hardware devices such as an intelligent robot. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0029] According to an embodiment of the present invention, an embodiment of a vestibular illusion training type verification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] In this embodiment, a vestibular illusion training type verification method is provided, which can be used for the above-mentioned electronic device. Figure 1 is a flow chart of a method for verifying vestibular illusion training type according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S101, obtaining a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type.

[0031] Specifically, the electronic device may receive a preset vestibular illusion training type input by a user, and then determine the training data corresponding to the preset vestibular illusion training type according to the corresponding relationship between the preset vestibular illusion training type and the training data.

[0032] Among them, the preset vestibular illusion training types are the names of different types of illusions, such as Coriolis-backward roll illusion, Coriolis-forward roll illusion, Coriolis-right roll illusion, body gravity illusion, body rotation illusion, etc.

[0033] Among them, different preset vestibular illusion training types correspond to different training data, such as setting the rotation axis, rotation direction, rotation speed, movement time, linear movement mode, etc.

[0034] For example, taking the triggering of the Coriolis-backward roll illusion as an example, the Coriolis illusion-backward roll illusion refers to the process in which the preset person is rotating clockwise around the zb axis of the body at a constant speed. If the preset person suddenly rotates around the xb axis of the body at the same time, the preset person will feel that the body is rotating backward around the yb axis of the body. Therefore, the training data field includes {(axis), (direction), (rotation speed), (duration), (start of rotation time) (axis), (direction), (rotation speed), (duration)}. For example, the training data correspondingly includes {(zb), (clockwise), (60° / s), (60s), (30s) (xb), (clockwise), (45° / s), (1s)}.

[0035] Step S102: Perform vestibular illusion type training on a preset person based on the training data, and obtain motion parameters corresponding to the preset person.

[0036] Specifically, the electronic device can drive the six-degree-of-freedom motion platform to perform linear motion and rotational motion based on the training data to perform vestibular illusion type training on the preset person. The electronic device can obtain the motion parameters corresponding to the preset person based on the motion sensor worn on the preset person.

[0037] Among them, the six-degree-of-freedom motion platform can rotate and linearly move along the three axes of the six-degree-of-freedom motion platform coordinate system xp, yp, and zp, and can provide complex six-degree-of-freedom comprehensive motion acceleration stimulation. It is stipulated that the origin Op of the six-degree-of-freedom motion platform coordinate system is located at the mass center of the six-degree-of-freedom motion platform, the xp axis of the six-degree-of-freedom motion platform coordinate system is located within the symmetry plane of the six-degree-of-freedom motion platform, and the forward direction is positive; the yp axis of the six-degree-of-freedom motion platform coordinate system is perpendicular to the symmetry plane of the six-degree-of-freedom motion platform where the xp axis is located, and the right direction is positive; the zp axis of the six-degree-of-freedom motion platform coordinate system is perpendicular to the xpOpyp plane, and the positive direction of the zp axis is determined to be downward according to the right-hand rule.

[0038] Step S103, performing data processing on the motion parameters to generate target data.

[0039] Specifically, the electronic device can check the integrity and accuracy of the motion parameters, and handle missing values, outliers, and erroneous data in the motion parameters. Then, the motion parameters are standardized or normalized to make different motion parameters comparable. Meaningful features are extracted from the original motion parameters. These features can include derived parameters such as calculated speed, acceleration, angular velocity, etc. Then, the motion parameters are analyzed using statistical methods or data mining techniques. For example, statistics such as mean, variance, correlation, etc. are calculated to understand the distribution and characteristics of the data. Then, the electronic device performs data fusion on the motion parameter data from multiple sources, integrating the multi-source motion parameters into a more comprehensive data set to generate target data.

[0040] This step will be described in detail below.

[0041] Step S104, identifying the target data based on a preset type recognition model, and determining a target vestibular illusion recognition type corresponding to the target data.

[0042] Specifically, the electronic device can input the target data into a preset type recognition model, and the preset type recognition model can recognize the target data and extract features, and determine the target vestibular illusion recognition type corresponding to the target data based on the extracted features.

[0043] Among them, the preset type recognition model can be a radial basis function (RBF) network, a feedforward neural network (FFNN), a convolutional neural network (CNN), a deconvolutional neural network (DN), a deep convolutional inverse graphics network (DCIGN), a generative adversarial network (GAN), a recurrent neural network (RNN), a long / short term memory network (LSTM), a deep residual network (DRN) and an extreme learning machine (ELM). The embodiment of the present application does not limit the preset type recognition model.

[0044] Step S105 , comparing the target vestibular illusion recognition type with the preset vestibular illusion training type.

[0045] Optionally, the electronic device may compare the target vestibular illusion recognition type with a preset vestibular illusion training type.

[0046] Optionally, the electronic device may also calculate the similarity between the target vestibular illusion recognition type and the preset vestibular illusion training type.

[0047] Step S106: verify the training data according to the comparison result.

[0048] Optionally, if the target vestibular illusion recognition type is consistent with the preset vestibular illusion training type, the electronic device determines that the correspondence between the training data and the preset vestibular illusion training type is accurate, thereby ensuring the accuracy of the training data in the training database and the preset vestibular illusion training type.

[0049] If the target vestibular illusion recognition type is inconsistent with the preset vestibular illusion training type, the electronic device can determine that the correspondence between the training data and the preset vestibular illusion training type is inaccurate, and then the electronic device can verify the training data to detect whether the acquired training data corresponding to the preset vestibular illusion training type is accurate.

[0050] Specifically, the electronic device can search for training data corresponding to the preset vestibular illusion training type in a database based on the preset vestibular illusion training type, and then compare the training data found in the database with the received training data corresponding to the preset vestibular illusion training type, and if the two are inconsistent, it is determined that the received training data is wrong, and the received training data is corrected based on the found training data.

[0051] If the found training data is consistent with the received training data corresponding to the preset vestibular illusion training type, it is determined that the training data corresponding to the preset vestibular illusion training type in the database is inaccurate, and the electronic device can adjust the training data. Optionally, the electronic device can adjust the parameters including axis and steering in the test data.

[0052] Under the condition that the adjusted training data is accurate, the training data corresponding to the preset vestibular illusion training type is saved in the database, so that the accuracy of the training data corresponding to the vestibular illusion training type in the database can be ensured.

[0053] The vestibular illusion training type verification method provided in the embodiment of the present application obtains a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type; performs vestibular illusion type training on a preset person based on the training data, and obtains motion parameters corresponding to the preset person, thereby ensuring the accuracy of the acquired motion parameters. Data processing is performed on the motion parameters to generate target data, thereby ensuring the accuracy of the generated target data. Based on a preset type recognition model, the target data is identified, and the target vestibular illusion recognition type corresponding to the target data is determined, thereby ensuring the accuracy of the target vestibular illusion recognition type corresponding to the determined target data. The target vestibular illusion recognition type is compared with the preset vestibular illusion training type to ensure the accuracy of the generated comparison result. According to the comparison result, the training data is verified to ensure the accuracy of the verification of the training data. Thus, the accuracy of the training data corresponding to the vestibular illusion training type can be guaranteed, thereby improving the accuracy of the vestibular illusion type training.

[0054] In this embodiment, a vestibular illusion training type verification method is provided, which can be used for the above-mentioned electronic device. Figure 2 is a flow chart of a method for verifying vestibular illusion training type according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps: Step S201, obtaining a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type.

[0055] Please refer to step S101 of the above embodiment for details, which will not be described again here.

[0056] Step S202: Perform vestibular illusion type training on a preset person based on the training data, and obtain motion parameters corresponding to the preset person.

[0057] Please refer to step S102 of the above embodiment for details, which will not be described again here.

[0058] Step S203, performing data processing on the motion parameters to generate target data.

[0059] Specifically, the target data is a target color image; the above step S203 includes: Step S2031, performing image conversion processing on the motion parameters to generate a grayscale image corresponding to the motion parameters.

[0060] In an optional implementation of the present application, before identifying the motion parameters, the electronic device may unify the sampling frequency of the motion parameters, then perform noise reduction processing, and finally normalize the data using the minimum-maximum method.

[0061] In an optional embodiment of the present application, the motion parameters include a head motion parameter sequence and a body motion parameter sequence. The motion sensor includes a head-mounted motion sensor and a body motion sensor. The head-mounted motion sensor can collect the head motion parameter sequence, and the body motion sensor can collect the body motion parameter sequence. The grayscale image includes a head grayscale image and a body grayscale image; the above step S2031 includes: Step a1, performing data conversion on the head motion parameter sequence to generate a head grayscale image.

[0062] Specifically, the electronic device can use phase space reconstruction and recursive graph technology to perform data conversion on the head motion parameter sequence to generate a head grayscale image, and perform data conversion on the body motion parameter sequence to generate a body grayscale image.

[0063] Specifically, the above step a1 may include the following steps: Step a11, splitting the head motion parameters to generate multiple non-intersecting head motion parameter subsequences.

[0064] Step a12, calculating the correlation integral between each head motion parameter subsequence according to the relationship between each head motion parameter subsequence.

[0065] Step a13, calculating the autocorrelation characteristics between the head motion parameter subsequences according to the correlation integral between the head motion parameter subsequences.

[0066] Step a14, determining the delay time and embedding dimension corresponding to the head motion parameter sequence according to the autocorrelation characteristics between each head motion parameter subsequence.

[0067] Step a15, generating a head phase space corresponding to the head motion parameter sequence based on the delay time and the embedding dimension.

[0068] Step a16, generating a head grayscale image according to the distance between each head vector in the head phase space.

[0069] Specifically, two key parameters of phase space reconstruction are the delay time and the embedding dimension. , the given time series A can represent a head motion parameter sequence, that is, a time series data of a segment of motion parameters.

[0070] In an optional embodiment, the head motion parameter sequence includes a linear acceleration motion parameter sequence of the head in the X, Y, and Z axes and an angular acceleration motion parameter sequence of the head in the X, Y, and Z axes; the head grayscale image includes an acceleration grayscale image of the head in the X-axis line, an acceleration grayscale image of the head in the Y-axis line, an acceleration grayscale image of the head in the Z-axis line, an angular acceleration grayscale image of the head in the X-axis, an angular acceleration grayscale image of the head in the Y-axis, and an angular acceleration grayscale image of the head in the Z-axis.

[0071] Specifically, the head-mounted motion sensor is used to detect the trainee's head movement and record the trainee's head three-dimensional linear acceleration (hx, hy, hz) and angular acceleration (hrx, hry, hrz). The head three-dimensional linear acceleration (hx, hy, hz) and angular acceleration (hrx, hry, hrz) are relative to the head coordinate system. It is stipulated that the origin Oh of the head coordinate system is located at the mass center of the head, the xh axis of the head coordinate system is located within the head symmetry plane, and the nose direction is positive; the yh axis of the head coordinate system is perpendicular to the head symmetry plane where the xh axis is located, and the right direction is positive; the zh axis of the head coordinate system is perpendicular to the xhOhyh plane, and the positive direction of the zh axis is determined to be downward according to the right-hand rule.

[0072] The body motion sensor is used to detect the motion parameters of the motion platform and record the three-dimensional linear acceleration (bx, by, bz) and three-dimensional angular position (bRx, bRy, bRz) of the trainee's body. The three-dimensional linear acceleration (bx, by, bz) and three-dimensional angular position (bRx, bRy, bRz) of the body are relative to the body coordinate system. It is stipulated that the origin Ob of the body coordinate system is located at the mass center of the head, the xb axis of the body coordinate system is located within the body symmetry plane, and the chest direction is positive; the yb axis of the body coordinate system is perpendicular to the body symmetry plane where the xb axis is located, and the right direction is positive; the zb axis of the body coordinate system is perpendicular to the xbObyb plane, and the positive direction of the zb axis is determined to be downward according to the right-hand rule.

[0073] The electronic device can perform data conversion on the linear acceleration motion parameter sequence of the head on the X-axis to generate a grayscale image of the acceleration of the head on the X-axis; perform data conversion on the linear acceleration motion parameter sequence of the head on the Y-axis to generate a grayscale image of the acceleration of the head on the Y-axis; perform data conversion on the linear acceleration motion parameter sequence of the head on the Z-axis to generate a grayscale image of the acceleration of the head on the Z-axis; perform data conversion on the angular acceleration motion parameter sequence of the head on the X-axis to generate a grayscale image of the angular acceleration of the head on the X-axis; perform data conversion on the angular acceleration motion parameter sequence of the head on the Y-axis to generate a grayscale image of the angular acceleration of the head on the Y-axis; perform data conversion on the angular acceleration motion parameter sequence of the head on the Z-axis to generate a grayscale image of the angular acceleration of the head on the Z-axis.

[0074] Specifically, phase space reconstruction can be performed using phase space B, as shown in formula (1).

[0075] (1) Where p is the embedding dimension, τ is the delay time, is the number of phase points in the phase space. By properly choosing the embedding dimension and the delay time, the reconstructed phase space becomes a diffeomorphism of the original system.

[0076] Among them, the electronic device can use the CC method to calculate the delay time τ and the embedding dimension p.

[0077] The specific process is as follows: First, split the time series A into τ non-intersecting subsequences, each with a length of . The value of τ can start from 1, and the maximum value is the smallest number between 40 and (N2 / 16-1), where N is the number of data points in time series A. As shown in formula (2).

[0078] (2) The sampling interval of time series A is , is the optimal delay time of the time series, is the delay time window, It can be calculated according to formula (3).

[0079] (3) The correlation integral is a cumulative distribution function that represents the probability that the distance between any two points in the phase space is less than a given search radius r. Distance refers to the distance between points in a high-dimensional space measured by the infinite norm of the difference between the vectors of the two points. Therefore, the correlation integral that defines the reconstructed phase space B is calculated according to formula (4).

[0080] (4) Where r is the search radius, , is the infinite norm between two subsequences in the reconstructed phase space B, , is the number of phase points in phase space. t is Defining the test statistic , calculated according to formula (5).

[0081] (5) The actual time series is of finite length and there is a certain correlation between the elements, resulting in the actual detection statistics Usually not zero. Test statistic Reflects the autocorrelation characteristics of each sub-time series, the optimal time delay Desirable The first zero of For the time point at which all radii r differ from each other the least, the points in the reconstructed phase space are closest to uniform distribution, and the orbit of the reconstructed attractor is fully expanded in the phase space. Select the two radii r with the largest and smallest corresponding values, and define the difference calculated according to formula (6). This time point is either the first zero point of a function or the time point at which all radii r differ from each other the least. At this time point, the points in the reconstructed phase space are closest to uniform distribution, and the orbit of the reconstructed attractor is fully expanded in the phase space.

[0082] (6) In formula (6), The detection statistic is measured The maximum deviation degree for all search radius r. So the local maximum time t should be The zero and The minimum value, optimal delay time corresponds to the first of these local maximum times t.

[0083] Then, the electronic device uses the CC method to calculate the delay time and embedding dimension. For example, N=320, p=2,3,4,5, , is the standard deviation of the time series, where i=1,2,3,4. Calculated according to formulas (7)-(9) , and .

[0084] (7) (8) (9)

[0085] In formulas (7)-(9), the CC method takes The first zero point or The first minimum value of , calculate the time delay τ = . And comprehensive and ,Pick The global minimum of the time series is taken as the time window length Finally, according to formula (3), the embedding dimension p is calculated. Then, according to formula (3), the delay time is calculated using the embedding dimension, and the phase space is calculated based on the embedding dimension and the delay time.

[0086] The recurrence graph is a two-dimensional graph consisting of black points, white points, and two time axes. After reconstructing the time series in the phase space, the phase space is obtained. .

[0087] Among them, Representation vector With vector If and The distance is less than a certain threshold ,but and are considered to be close to each other, and the corresponding position Nij on the recurrence graph is marked as 1 (indicated by a black dot); otherwise, it is marked as 0 (indicated by a white dot), as shown in formula (10).

[0088] (10) In formula (10), , is the distance threshold, is the Heaviside function, calculated according to formula (11).

[0089] (11) Nij is calculated according to formula (12).

[0090] (12) Step a2, performing data conversion on the body motion parameter sequence to generate a body grayscale image.

[0091] Specifically, the body motion parameter sequence includes the linear acceleration motion parameter sequence of the body in the X, Y, and Z axes and the angular acceleration motion parameter sequence of the body in the X, Y, and Z axes. The body grayscale image includes the body acceleration grayscale image in the X axis, the body acceleration grayscale image in the Y axis, the body acceleration grayscale image in the Z axis, the body angular acceleration grayscale image in the X axis, the body angular acceleration grayscale image in the Y axis, and the body angular acceleration grayscale image in the Z axis.

[0092] The electronic device can perform data conversion on the linear acceleration motion parameter sequence of the body on the X-axis to generate a grayscale image of the body's acceleration on the X-axis; perform data conversion on the linear acceleration motion parameter sequence of the body on the Y-axis to generate a grayscale image of the body's acceleration on the Y-axis; perform data conversion on the linear acceleration motion parameter sequence of the body on the Z-axis to generate a grayscale image of the body's acceleration on the Z-axis; perform data conversion on the angular acceleration motion parameter sequence of the body on the X-axis to generate a grayscale image of the body's angular acceleration on the X-axis; perform data conversion on the angular acceleration motion parameter sequence of the body on the Y-axis to generate a grayscale image of the body's angular acceleration on the Y-axis; perform data conversion on the angular acceleration motion parameter sequence of the body on the Z-axis to generate a grayscale image of the body's angular acceleration on the Z-axis.

[0093] The data conversion method can be found above and will not be described in detail here.

[0094] Step S2032, generating a target color image based on the grayscale image.

[0095] In some optional implementations, the grayscale image includes a head grayscale image and a body grayscale image, and the above step S2032 includes: Step b1, generating a color image of the head based on the grayscale image of the head.

[0096] Specifically, the grayscale image of the head includes the grayscale image of the head acceleration on the X-axis, the grayscale image of the head acceleration on the Y-axis, the grayscale image of the head acceleration on the Z-axis, the grayscale image of the head angular acceleration on the X-axis, the grayscale image of the head angular acceleration on the Y-axis, and the grayscale image of the head angular acceleration on the Z-axis; the color image of the head includes the color image of the head linear acceleration and the color image of the head angular acceleration.

[0097] Specifically, the above step b1 may include the following steps: Step b11, determining a first red channel from the grayscale image of the acceleration of the head on the X axis, the grayscale image of the acceleration of the head on the Y axis, and the grayscale image of the acceleration of the head on the Z axis, determining a first green channel from the two grayscale images except the first red channel, and determining the last grayscale image as the first blue channel; Step b12, generating a head linear acceleration color image according to the determined first red channel, first green channel and first blue channel; Step b13, determining a second red channel from the grayscale image of the angular acceleration of the head on the X-axis, the grayscale image of the angular acceleration of the head on the Y-axis, and the grayscale image of the angular acceleration of the head on the Z-axis, determining a second green channel from the two grayscale images except the second red channel, and determining the last grayscale image as the second blue channel; Step b14, generating a head angular acceleration color image according to the determined second red channel, second green channel and second blue channel.

[0098] Specifically, the electronic device can use the grayscale image of the head acceleration on the X-axis, the grayscale image of the head acceleration on the Y-axis, and the grayscale image of the head acceleration on the Z-axis as the red, green, and blue channels of the head linear acceleration color image, respectively, to obtain a head linear acceleration color image.

[0099] The electronic device can use the grayscale image of the head angular acceleration on the X-axis, the grayscale image of the head angular acceleration on the Y-axis, and the grayscale image of the head angular acceleration on the Z-axis as the red, green, and blue channels of the head angular acceleration color image, respectively, to obtain a head angular acceleration color image.

[0100] It should be noted that the embodiment of the present application does not specifically limit the correspondence between the red, green and blue channels of the head X-axis angular acceleration grayscale image, the head Y-axis angular acceleration grayscale image, and the head Z-axis angular acceleration grayscale image as color images, respectively.

[0101] Step b2, generating a body color image based on the body grayscale image.

[0102] Specifically, the body grayscale image includes the body's X-axis acceleration grayscale image, the body's Y-axis acceleration grayscale image, the body's Z-axis acceleration grayscale image, the body's X-axis angular acceleration grayscale image, the body's Y-axis angular acceleration grayscale image, and the body color image includes the body's linear acceleration color image and the body's angular acceleration color image.

[0103] Specifically, the above step b2 may include the following steps: Step b21, determining a first red channel from the grayscale image of the acceleration of the body on the X axis, the grayscale image of the acceleration of the body on the Y axis, and the grayscale image of the acceleration of the body on the Z axis, determining a first green channel from the two grayscale images except the first red channel, and determining the last grayscale image as the first blue channel; Step b22, generating a body linear acceleration color image according to the determined first red channel, first green channel and first blue channel; Step b23, determining a second red channel from the grayscale image of the body's angular acceleration on the X-axis, the grayscale image of the body's angular acceleration on the Y-axis, and the grayscale image of the body's angular acceleration on the Z-axis, determining a second green channel from the two grayscale images other than the second red channel, and determining the last grayscale image as the second blue channel; Step b24, generating a body angular acceleration color image according to the determined second red channel, second green channel and second blue channel.

[0104] Step b3, splicing the head color image and the body color image to generate a target color image.

[0105] Specifically, the electronic device can use the grayscale image of the body's acceleration on the X-axis, the grayscale image of the body's acceleration on the Y-axis, and the grayscale image of the body's acceleration on the Z-axis as red, green, and blue channels of a body linear acceleration color image, respectively, to obtain a body linear acceleration color image.

[0106] The electronic device can use the grayscale image of the body's angular acceleration on the X-axis, the grayscale image of the body's angular acceleration on the Y-axis, and the grayscale image of the body's angular acceleration on the Z-axis as the red, green, and blue channels of a body angular acceleration color image, respectively, to obtain a body angular acceleration color image.

[0107] It should be noted that the embodiment of the present application does not specifically limit the correspondence between the red, green and blue channels of the body's X-axis angular acceleration grayscale image, the body's Y-axis angular acceleration grayscale image, and the body's Z-axis angular acceleration grayscale image as color images, respectively.

[0108] Step S204: identifying the target data based on a preset type recognition model, and determining a target vestibular illusion recognition type corresponding to the target data.

[0109] For details, please see Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0110] Step S205 , comparing the target vestibular illusion recognition type with the preset vestibular illusion training type.

[0111] For details, please see Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0112] Step S206: verify the training data according to the comparison result.

[0113] For details, please see Figure 1 Step S106 of the illustrated embodiment will not be described in detail here.

[0114] For example, Figure 3As shown, the electronic device converts the linear acceleration motion parameter sequence of the head in the X, Y, and Z axes and the angular acceleration motion parameter sequence of the head in the X, Y, and Z axes to generate the head acceleration grayscale image in the X axis, the head acceleration grayscale image in the Y axis, the head acceleration grayscale image in the Z axis, the head angular acceleration grayscale image in the X axis, the head angular acceleration grayscale image in the Y axis, and the head angular acceleration grayscale image in the Z axis. The electronic device converts the linear acceleration motion parameter sequence of the body in the X, Y, and Z axes and the angular acceleration motion parameter sequence of the body in the X, Y, and Z axes to generate the body acceleration grayscale image in the X axis, the body acceleration grayscale image in the Y axis, the body acceleration grayscale image in the Z axis, the body angular acceleration grayscale image in the X axis, the body angular acceleration grayscale image in the Y axis, and the body angular acceleration grayscale image in the Z axis.

[0115] Then, the electronic device transforms the grayscale image of the head acceleration on the X-axis, the grayscale image of the head acceleration on the Y-axis, and the grayscale image of the head acceleration on the Z-axis to generate a head linear acceleration color image; transforms the grayscale image of the head angular acceleration on the X-axis, the grayscale image of the head angular acceleration on the Y-axis, and the grayscale image of the head acceleration on the Z-axis to generate a head angular acceleration color image; transforms the grayscale image of the body acceleration on the X-axis, the grayscale image of the body acceleration on the Y-axis, and the grayscale image of the body acceleration on the Z-axis to generate a body linear acceleration color image; transforms the grayscale image of the body angular acceleration on the X-axis, the grayscale image of the body angular acceleration on the Y-axis, and the grayscale image of the body acceleration on the Z-axis to generate a body angular acceleration color image.

[0116] The electronic device splices the head linear acceleration color image, the head angular acceleration color image, the body linear acceleration color image and the body angular acceleration color image to generate a target color image.

[0117] The vestibular illusion training type verification method provided in the embodiment of the present application splits the head motion parameters to generate multiple non-intersecting head motion parameter subsequences; according to the relationship between each head motion parameter subsequence, the correlation integral between each head motion parameter subsequence is calculated to ensure the accuracy of the calculated correlation integral between the head motion parameter subsequences. According to the correlation integral between each head motion parameter subsequence, the autocorrelation characteristics between each head motion parameter subsequence are calculated to ensure the accuracy of the calculated autocorrelation characteristics between the head motion parameter subsequences. According to the autocorrelation characteristics between each head motion parameter subsequence, the delay time and embedding dimension corresponding to the head motion parameter sequence are determined to ensure the accuracy of the delay time and embedding dimension corresponding to the determined head motion parameter sequence. Based on the delay time and embedding dimension, the head phase space corresponding to the head motion parameter sequence is generated to ensure the accuracy of the head phase space corresponding to the generated head motion parameter sequence. According to the distance between each head vector in the head phase space, a head grayscale image is generated to ensure the accuracy of the generated head grayscale image. Similarly, the body motion parameter sequence is converted to generate a body grayscale image to ensure the accuracy of the generated body grayscale image.

[0118] Among them, the head motion parameter sequence includes the linear acceleration motion parameter sequence of the head in the X, Y and Z axes and the angular acceleration motion parameter sequence of the head in the X, Y and Z axes respectively; the head grayscale image includes the acceleration grayscale image of the head in the X-axis line, the acceleration grayscale image of the head in the Y-axis line, the acceleration grayscale image of the head in the Z-axis line, the angular acceleration grayscale image of the head in the X-axis, the angular acceleration grayscale image of the head in the Y-axis, and the angular acceleration grayscale image of the head in the Z-axis.

[0119] The first red channel is determined from the grayscale image of the acceleration of the head on the X axis, the grayscale image of the acceleration of the head on the Y axis, and the grayscale image of the acceleration of the head on the Z axis, the first green channel is determined from the two grayscale images except the first red channel, and the last grayscale image is determined as the first blue channel; based on the determined first red channel, first green channel, and first blue channel, a head linear acceleration color image is generated, ensuring that the generated head linear acceleration color image can represent the linear acceleration motion parameter sequence of the head on the X, Y, and Z axes. The second red channel is determined from the grayscale image of the angular acceleration of the head on the X axis, the grayscale image of the angular acceleration of the head on the Y axis, and the grayscale image of the angular acceleration of the head on the Z axis, the second green channel is determined from the two grayscale images except the second red channel, and the last grayscale image is determined as the second blue channel; based on the determined second red channel, second green channel, and second blue channel, a head angular acceleration color image is generated. It is ensured that the generated head angular acceleration color image can represent the angular acceleration motion parameter sequence of the head on the X, Y, and Z axes. Similarly, based on the body grayscale image, a body color image is generated, which ensures the accuracy of the generated body color image. The head color image and the body color image are spliced ​​to generate a target color image, which ensures the accuracy of the generated target color image.

[0120] In this embodiment, a vestibular illusion training type verification method is provided, which can be used for the above-mentioned electronic device. Figure 4 is a flow chart of a method for verifying vestibular illusion training type according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps: Step S301, obtaining a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type.

[0121] Please refer to step S201 of the above embodiment for details, which will not be repeated here.

[0122] Step S302: Perform vestibular illusion type training on a preset person based on the training data, and obtain motion parameters corresponding to the preset person.

[0123] Please refer to step S202 of the above embodiment for details, which will not be repeated here.

[0124] Step S303, performing data processing on the motion parameters to generate target data.

[0125] Please refer to step S203 of the above embodiment for details, which will not be described again here.

[0126] Step S304: Identify the target data based on a preset type recognition model to determine the target vestibular illusion recognition type corresponding to the target data.

[0127] Specifically, the target data is a target color image, and the above step S304 may include the following steps: Step S3041, inputting the target color image into a preset type recognition model.

[0128] Specifically, the electronic device may input the target color image into a preset type recognition model.

[0129] Step S3042: The preset type recognition model performs feature recognition and feature extraction on the target color image, and outputs the target vestibular illusion recognition type based on the extracted features.

[0130] Specifically, the preset type recognition model includes a partitioning layer, a Swin Transformer layer (hierarchical visual transformer layer), a pooling layer, a fully connected layer and a Softmax layer. The above step S3042 may include the following steps: Step c1, dividing the target color image by layers to generate a plurality of target color image blocks, and generating label information corresponding to each target color image block.

[0131] Specifically, the architecture of the preset image recognition model is as shown in the attached Figure 5 The entire preset image recognition model includes a block partitioning layer, a Swin Transformer layer, a pooling layer, a fully connected layer, and a Softmax layer.

[0132] The division layer in the preset type recognition model divides the target color image to generate a plurality of target color image blocks, and generates label information corresponding to each target color image block.

[0133] Exemplarily, assume that the size of the target color image obtained by phase space reconstruction, recursive graph and image stitching is W×H×3, where W and H are the width and length of the target color image, respectively, and "3" represents the three color channels of the target color image. First, the division layer in the preset type recognition model passes the target color image through the block division layer. In the block division layer, the target color image is divided into "blocks", and the blocks are used as the smallest units instead of pixels. Each block corresponds to a "marker", and the number of marks is the same as the number of blocks, and the output forms a three-dimensional matrix of H / 4×W / 4.

[0134] In step c2, the Swin Transformer layer fuses the feature information corresponding to each target color image block to generate a fused feature, and performs a linear transformation on the channel data of each pixel in the fused feature to generate a multi-dimensional feature; and performs feature extraction on the multi-dimensional feature to generate a target feature.

[0135] Specifically, Figure 6As shown, the Swin Transformer layer includes four stages. The first stage includes the first normalization layer (layer normalization, LN) and the window multi-head self-attention layer (window multi-head self-attention, W-MSA), the second stage includes the second normalization layer (layer normalization, LN) and the first multi-layer perceptron layer (the multi-layer perceptron, MLP), the third stage includes the third normalization layer (layernormalization, LN) and the shifted window-based multi-head self-attention layer (shifted window-based multi-headself-attention, SW-MSA), and the fourth stage includes the fourth normalization layer (layer normalization, LN) and the second multi-layer perceptron layer (the multi-layer perceptron, MLP); the above step c2 includes: Step c21: The first normalization layer normalizes the fused features to generate first normalized features.

[0136] In step c22, the window multi-head self-attention layer performs self-attention calculation on the first normalized feature and performs residual calculation to generate the first attention feature.

[0137] Step c23, the second normalization layer normalizes the first attention feature to generate a second normalized feature.

[0138] In step c24, the first multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the second normalized feature to generate a multi-dimensional feature.

[0139] Step c25: The third normalization layer normalizes the multi-dimensional features to generate third normalized features.

[0140] Step c26, performing self-attention calculation on the third normalized feature based on the multi-head self-attention layer of the shifted window to generate a second attention feature.

[0141] Step c27, the fourth normalization layer normalizes the second attention feature to generate a fourth normalized feature.

[0142] In step c28, the second multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the fourth normalized feature to generate a target feature.

[0143] Specifically, in the first stage, the first normalization layer calculates the mean and variance of each sample feature of the fused feature, and normalizes the feature to generate the first normalized feature. During the training process, the parameters of the first normalization layer are adaptively adjusted according to the distribution of the data. Then, the window multi-head self-attention layer captures the feature dependencies within the local window in the input feature map. It performs weighted fusion of features by calculating the attention weights of each position with other positions in the window, so as to better model the interaction of local features. Specifically, the input feature map is divided into multiple non-overlapping windows, and self-attention is calculated in each window. Through the multi-head mechanism, multiple attention heads are calculated in parallel to obtain different feature representations. Finally, the output of each window is spliced ​​or merged to generate the first attention feature.

[0144] In the second stage, the second normalization layer calculates the mean and variance of each sample feature of the first attention feature, and normalizes the feature to generate the second normalized feature. The first multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the second normalized feature to generate a multi-dimensional feature.

[0145] Among them, the MLP layer usually consists of an input layer, multiple hidden layers, and an output layer. In each neuron, the input signal is linearly combined with the weights and nonlinearly transformed by the activation function. Information is transmitted between layers, and the weights are adjusted through the back-propagation algorithm to optimize the performance of the model.

[0146] In the third stage, the third normalization layer calculates the mean and variance of each sample feature for the multi-dimensional features, and normalizes the features to generate the third normalized features. Similar to W-MSA, the multi-head self-attention layer based on the shifted window first divides the input feature map into windows, but in adjacent layers, the windows are shifted to a certain extent. This allows a certain overlap between adjacent windows, so that more global information can be obtained. By calculating the self-attention within the shifted window, the features are modeled and fused to generate the second attention feature.

[0147] In the third stage, the fourth normalization layer normalizes the second attention feature to generate the fourth normalized feature. The second multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the fourth normalized feature to generate the target feature.

[0148] For example, as shown in the attached Figure 5As shown, during the first stage, the number of tags is kept W / 4×H / 4, the size of the output feature is set to D, and stage 1 is repeated twice; during the second stage, the number of tags is reduced to 1 / 4 of the previous one, becoming W / 8×H / 8, the output size is 2D, and stage 2 is repeated twice; the third and fourth stages also follow a similar pattern, the number of tags they output is W / 16×H / 16 and W / 32×H / 32 respectively, the output size is 4D and 8D respectively, and stage 3 and stage 4 are repeated 6 times and 2 times respectively.

[0149] For example, the four stages include 2, 2, 6, and 2 Swin Transformers, respectively. The number of channels in the four stages are 96, 192, 384, and 768, respectively. The number of attention heads included in each stage is 3, 6, 12, and 24, respectively.

[0150] The role of the block fusion layer in the Swin Transformer layer is to downsample, reduce resolution, and adjust the number of channels to establish a hierarchical structure. The features of each block group are concatenated in the first block fusion layer. The concatenated features are then input into the linear embedding layer for processing. The linear embedding layer performs a linear transformation on the channel data of each pixel and projects the channel data to an arbitrary dimension. The dimension after projection is recorded as "D", followed by feature processing in the Swin Transformer module.

[0151] Specifically, the structure of the SW-MSA layer is similar to that of the W-MSA layer, except for the feature calculation part, which involves a sliding window operation. The calculations of each part of the network structure are shown in formulas (13)-(16).

[0152] (13) (14) (15) (16) in, and They represent the output features of W-MSA and MLP for l, respectively, where l is the index of the partition.

[0153] The use of MSA improves accuracy and improves loss, thereby enhancing the generalization ability of the model. However, MSA has high computational complexity. By adopting W-MSA, the computational complexity can be reduced. The standard MSA calculation is shown in formulas (17)-(20), assuming that there are i heads and the number of blocks is w×h.

[0154] (17) (18) (19) (20) In formula (17)-(20), is the input matrix, are query matrix, key matrix and value matrix respectively, They are query parameter matrix, key parameter matrix and value parameter matrix respectively. is the transformation matrix, s is the dimension of the query matrix and the key matrix, Represents the connection matrix.

[0155] The self-attention calculation of each window is shown in formula (21).

[0156] (twenty one) In formula (21) are the query matrix, key matrix and value matrix respectively, where is the number of blocks in the current window, s is the dimension of the query matrix and the key matrix. F is the bias matrix Obtained.

[0157] The calculation of each window of MSA and W-MSA is shown in formulas (22)-(24).

[0158] (twenty two) (twenty three) (twenty four) In formulas (22)-(24), D represents the dimension of the feature. P represents the length and width of each window (both length and width are P).

[0159] Step c3: The pooling layer compresses and extracts the target features to generate local features.

[0160] Specifically, the electronic device may obtain the size and step size of the pooling window in the pooling layer. Then, based on the pooling window size and step size, slide on the input target feature map. For each pooling window position, calculation is performed according to the selected pooling method.

[0161] Optionally, if the pooling layer is max pooling, find the maximum value in the window as the output value at that position. If the pooling layer is average pooling, calculate the average of all values ​​in the window as the output value at that position.

[0162] Then, after the pooling operation, the output feature map is the extracted local features. These local features retain the main information of the original target features, while reducing the dimension of the feature map and reducing the computational complexity of subsequent processing.

[0163] In step c4, the fully connected layer integrates local features to generate global features and maps the global features to output categories.

[0164] Specifically, the fully connected layer can perform a weighted combination of local features. Each connection has a corresponding weight, and by learning these weights, the fully connected layer can automatically determine how to best integrate local features to form more representative global features, and then the generated global features are further mapped to the output category.

[0165] In step c5, the Softmax layer converts the output category of the fully connected layer output into a probability distribution, and determines the target vestibular illusion recognition type based on the probability distribution.

[0166] Specifically, the Softmax layer converts the output category of the fully connected layer output into a probability distribution, and distributes the probability, and determines the type with the largest probability distribution as the target vestibular illusion recognition type.

[0167] Step S305 : comparing the target vestibular illusion recognition type with the preset vestibular illusion training type.

[0168] Please refer to step S205 of the above embodiment for details, which will not be repeated here.

[0169] Step S306: verify the training data according to the comparison result.

[0170] Please refer to step S206 of the above embodiment for details, which will not be repeated here.

[0171] The vestibular illusion training type verification method provided in the embodiment of the present application inputs the target color image into the preset type recognition model; the division layer divides the target color image to generate multiple target color image blocks, and generates label information corresponding to each target color image block; the first normalization layer normalizes the fused features to generate the first normalized features, so that the preset type recognition model has stronger robustness to the scale change of the input. The window multi-head self-attention layer performs self-attention calculation on the first normalized feature, and performs residual calculation to generate the first attention feature, so that the multi-head mechanism can be used to pay attention to multiple different representation subspaces at the same time, thereby improving the learning ability and generalization performance of the preset type recognition model. The second normalization layer normalizes the first attention feature to generate the second normalized feature; the first multi-layer perceptron layer performs linear transformation on the channel data of each pixel of the second normalized feature to generate multi-dimensional features, which increases the expression ability of the preset type recognition model. The third normalization layer normalizes the multi-dimensional features to generate the third normalized features; the multi-head self-attention layer based on the shift window performs self-attention calculation on the third normalized features to generate the second attention features, which helps to better model long-distance dependencies and further improve the feature representation ability of the preset type recognition model. The fourth normalization layer normalizes the second attention features to generate the fourth normalized features; the second multi-layer perceptron layer performs linear transformation on the channel data of each pixel of the fourth normalized features to generate the target features, ensuring the accuracy of the generated target features. The pooling layer compresses and extracts the target features to generate local features, ensuring the accuracy of the generated local features. The fully connected layer integrates the local features to generate global features and maps the global features to the output categories; the Softmax layer converts the output categories of the fully connected layer output into probability distributions, and determines the target vestibular illusion recognition type based on the probability distribution, ensuring the accuracy of the determined target vestibular illusion recognition type.

[0172] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A vestibular illusion training type verification method, characterized in that: The method comprises: Obtaining a preset vestibular illusion training type and training data corresponding to the preset vestibular illusion training type; Based on the training data, vestibular illusion type training is performed on a preset person, and motion parameters corresponding to the preset person are obtained; Performing data processing on the motion parameters to generate target data; Based on a preset type recognition model, the target data is recognized to determine a target vestibular illusion recognition type corresponding to the target data; comparing the target vestibular illusion recognition type with the preset vestibular illusion training type; According to the comparison result, the training data is verified.

2. The method according to claim 1, characterized in that: The target data is a target color image; the step of performing data processing on the motion parameters to generate the target data includes: Performing image conversion processing on the motion parameters to generate a grayscale image corresponding to the motion parameters; Based on the grayscale image, the target color image is generated.

3. The method according to claim 2, characterized in that The motion parameters include a head motion parameter sequence and a body motion parameter sequence, and the grayscale image includes a head grayscale image and a body grayscale image; performing image conversion processing on the motion parameters to generate a grayscale image corresponding to the motion parameters includes: Performing data conversion on the head motion parameter sequence to generate the head grayscale image; The body motion parameter sequence is subjected to data conversion to generate the body grayscale image.

4. The method according to claim 3, characterized in that The step of performing data conversion on the head motion parameter sequence to generate a head grayscale image includes: Splitting the head motion parameter to generate a plurality of mutually non-intersecting head motion parameter subsequences; Calculating the correlation integral between the head motion parameter subsequences according to the relationship between the head motion parameter subsequences; Calculating the autocorrelation characteristics between the head motion parameter subsequences according to the correlation integral between the head motion parameter subsequences; Determine the delay time and embedding dimension corresponding to the head motion parameter sequence according to the autocorrelation characteristics between each of the head motion parameter subsequences; Generate a head phase space corresponding to the head motion parameter sequence based on the delay time and the embedding dimension; The head grayscale image is generated according to the distances between the head vectors in the head phase space.

5. The method according to claim 4, characterized in that The head motion parameter sequence includes a linear acceleration motion parameter sequence of the head in the X, Y, and Z axes and an angular acceleration motion parameter sequence of the head in the X, Y, and Z axes respectively; the head grayscale image includes an acceleration grayscale image of the head in the X-axis line, an acceleration grayscale image of the head in the Y-axis line, an acceleration grayscale image of the head in the Z-axis line, an angular acceleration grayscale image of the head in the X-axis, an angular acceleration grayscale image of the head in the Y-axis, and an angular acceleration grayscale image of the head in the Z-axis.

6. The method according to claim 2, characterized in that The grayscale image includes a head grayscale image and a body grayscale image, and generating the target color image based on the grayscale image includes: Based on the head grayscale image, generate a head color image; Based on the body grayscale image, generate a body color image; The head color image and the body color image are spliced ​​to generate the target color image.

7. The method according to claim 6, characterized in that The head grayscale image includes an acceleration grayscale image of the head on the X axis, an acceleration grayscale image of the head on the Y axis, an acceleration grayscale image of the head on the Z axis, an angular acceleration grayscale image of the head on the X axis, an angular acceleration grayscale image of the head on the Y axis, and an angular acceleration grayscale image of the head on the Z axis; the head color image includes a head linear acceleration color image and a head angular acceleration color image; generating a head color image based on the head grayscale image includes: Determine a first red channel from the grayscale image of the acceleration of the head on the X axis, the grayscale image of the acceleration of the head on the Y axis, and the grayscale image of the acceleration of the head on the Z axis, determine a first green channel from the two grayscale images except the first red channel, and determine the last grayscale image as a first blue channel; generating the head linear acceleration color image according to the determined first red channel, the first green channel and the first blue channel; Determine a second red channel from the grayscale image of the angular acceleration of the head on the X-axis, the grayscale image of the angular acceleration of the head on the Y-axis, and the grayscale image of the angular acceleration of the head on the Z-axis, determine a second green channel from the two grayscale images except the second red channel, and determine the last grayscale image as the second blue channel; The head angular acceleration color image is generated according to the determined second red channel, the second green channel, and the second blue channel.

8. The method according to claim 1, characterized in that The target data is a target color image, and the target data is identified based on a preset type recognition model to determine the target vestibular illusion recognition type corresponding to the target data, including: Inputting the target color image into the preset type recognition model; The preset type recognition model performs feature recognition and feature extraction on the target color image, and outputs the target vestibular illusion recognition type based on the extracted features.

9. The method according to claim 8, characterized in that The preset type recognition model includes a partitioning layer, a SwinTransformer layer, a pooling layer, a fully connected layer and a Softmax layer. The preset type recognition model performs feature recognition and feature extraction on the target color image, and outputs the target vestibular illusion recognition type based on the extracted features, including: The division layer divides the target color image to generate a plurality of target color image blocks, and generates label information corresponding to each of the target color image blocks; The Swin Transformer layer fuses the feature information corresponding to each of the target color image blocks to generate a fused feature, and performs a linear transformation on the channel data of each pixel in the fused feature to generate a multi-dimensional feature; and performs feature extraction on the multi-dimensional feature to generate a target feature; The pooling layer compresses and extracts the target features to generate local features; The fully connected layer integrates the local features to generate global features, and maps the global features to output categories; The Softmax layer converts the output category of the fully connected layer into a probability distribution, and determines the target vestibular illusion recognition type based on the probability distribution.

10. The method according to claim 9, characterized in that The Swin Transformer layer includes four stages, the first stage includes a first normalization layer and a window multi-head self-attention layer, the second stage includes a second normalization layer and a first multi-layer perceptron layer, the third stage includes a third normalization layer and a multi-head self-attention layer based on a shifted window, and the fourth stage includes a fourth normalization layer and a second multi-layer perceptron layer; the Swin Transformer layer fuses the feature information corresponding to each of the target color image blocks to generate a fused feature, and performs a linear transformation on the channel data of each pixel in the fused feature to generate a multi-dimensional feature; Extracting the multi-dimensional features to generate target features includes: The first normalization layer normalizes the fused features to generate first normalized features; The window multi-head self-attention layer performs self-attention calculation on the first normalized feature, and performs residual calculation to generate a first attention feature; The second normalization layer normalizes the first attention feature to generate a second normalized feature; The first multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the second normalized feature to generate the multi-dimensional feature; The third normalization layer normalizes the multi-dimensional features to generate third normalized features; The multi-head self-attention layer based on the shifted window performs self-attention calculation on the third normalized feature to generate a second attention feature; The fourth normalization layer normalizes the second attention feature to generate a fourth normalized feature; The second multi-layer perceptron layer performs a linear transformation on the channel data of each pixel of the fourth normalized feature to generate the target feature.

Images