Wearable perception interaction system and interaction method based on augmented reality game

Abstract

The invention provides a wearable perception interaction method based on an augmented reality game. The wearable perception interaction method comprises the steps that a controller obtains first training data based on a first sensor; the controller obtains second training data based on the second sensor; the controller inputs the first training data and the second training data in the training period into a first neural network prediction model; the controller obtains first actual data based on the first sensor; the controller inputs the first actual data in the first use period into the first neural network training model, and outputs a result according to the first neural network model; and the controller takes the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion attitude parameter as data of the second sensor in the first use period and displays the data. The invention provides a wearable perception interaction system based on an augmented reality game and electronic equipment.

Description

Technical Field

[0001] The present application relates to the field of interactive systems, and in particular to a wearable sensing interactive system and an interactive method based on augmented reality games. Background Art

[0002] With the rapid development of science and technology, we are gradually entering a new era full of infinite possibilities. In this era, augmented reality (AR) technology has gradually become a shining star in the field of science and technology with its unique charm. It has not only changed our lifestyle, but also brought us an unprecedented visual experience.

[0003] Among the application scenarios of augmented reality technology, games and entertainment are undoubtedly one of the most attractive areas. By wearing AR glasses or mobile phones and other terminal devices, you can be in a world full of fantasy, interact with virtual characters, and experience unprecedented game fun.

[0004] An augmented reality interactive device in the gaming field includes VR glasses and sensors for the left and right hands. In order to improve the wearing experience of the sensors for the left and right hands, the size of the sensors is small, so the battery is small and the battery life is short. Users need to charge them frequently, and the user experience is poor. Summary of the invention

[0005] The embodiments of the present application provide a wearable perception interaction system and interaction method based on augmented reality games to improve the above-mentioned problems.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In the first aspect, the embodiment of the present application proposes a wearable perception interaction method based on an augmented reality game, the method is applicable to a wearable sensing device, the wearable device includes a first motion sensor worn on the left hand, a second motion sensor worn on the left hand, and a controller, the method is applicable to the controller, and the method includes:

[0008] The controller acquires first training data based on the first sensor, the first training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle;

[0009] The controller acquires second training data based on the second sensor, the second training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor during a training cycle;

[0010] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter;

[0011] The controller acquires first actual data based on the first sensor, the first actual data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor in a first use cycle;

[0012] The controller inputs the first actual data in the first use cycle into the first neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the first neural network model;

[0013] The controller uses the predicted movement direction parameter, the predicted movement speed parameter and the predicted movement posture parameter as the data of the second sensor in the first use cycle and displays them.

[0014] In combination with the first aspect, in some embodiments, before the controller acquires first training data based on the first sensor, the first training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle, the method further includes:

[0015] The controller determines the current game scene as the target scene;

[0016] The controller uses the predicted motion direction parameter, the predicted motion speed parameter, and the predicted motion posture parameter as data of the second sensor in the first use cycle and displays them, including:

[0017] Determine whether the current game scene is the target scene. If the current game scene is the target scene, the predicted motion speed parameter and the predicted motion posture parameter are used as the data of the second sensor in the first use cycle and displayed.

[0018] In conjunction with the first aspect, in some embodiments, the method further comprises:

[0019] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the second neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter;

[0020] The controller acquires second actual data based on the second sensor, the second actual data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor in a second use cycle;

[0021] The controller inputs the second actual data in the second use cycle into the second neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the second neural network model;

[0022] The controller uses the predicted movement direction parameter, the predicted movement speed parameter and the predicted movement posture parameter as the data of the first sensor in the second use cycle and displays them.

[0023] In conjunction with the first aspect, in some implementations, the first usage period does not overlap with the second usage period.

[0024] In combination with the first aspect, in some embodiments, the controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter, and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter, and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter, and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter, and motion posture parameter, including:

[0025] The controller takes the motion direction parameters, motion speed parameters and motion posture parameters in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameters, motion speed parameters and motion posture parameters of the second training data and perform iterative calculations. When the preset conditions are met, the calculation is stopped and the trained first neural network prediction model is output.

[0026] In combination with the first aspect, in some embodiments, the controller uses the motion direction parameter, motion speed parameter, and motion posture parameter in the first training data as three different dimensions, and the three dimensions respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter, and motion posture parameter of the second training data and perform iterative calculations, and stops calculating and outputting the trained first neural network prediction model when a preset condition is met, including:

[0027] Each dimension is normalized so that it lies in the range [0,1].

[0028] In combination with the first aspect, in some embodiments, the controller uses the motion direction parameter, motion speed parameter, and motion posture parameter in the first training data as three different dimensions, and the three dimensions respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter, and motion posture parameter of the second training data and perform iterative calculations, and stops calculating and outputting the trained first neural network prediction model when a preset condition is met, including:

[0029] The controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, and compares the error value t1, the error value t3, and the error value t3 with the preset error value j1, the preset error value j2, and the preset error value j2, respectively, and satisfies:

[0030]

[0031] Where K is a constant.

[0032] In combination with the first aspect, in some implementations, the controller obtains an error value t1 corresponding to the motion direction parameter, an error value t2 corresponding to the motion speed parameter, and an error value t3 corresponding to the motion posture parameter, including:

[0033] The error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter satisfy:

[0034] t1=(x0-x v )x v ;

[0035] t2=(y0-y v )y v ;

[0036] t3=(z0-z v ) v ;

[0037] Among them, x0, y0, z0 are the values ​​corresponding to the motion direction parameter, motion speed parameter, and motion posture parameter obtained by the second sensor in the first use period, respectively. v ,y v 、z v They are respectively predicted movement direction parameters, predicted movement speed parameters and predicted movement posture parameters output by the first neural network prediction model according to the first training data.

[0038] In a second aspect, an embodiment of the present application proposes a wearable perception interaction system based on an augmented reality game, the system comprising a first motion sensor worn on the left hand, a second motion sensor worn on the left hand, and a controller, the method is applicable to the controller, and the system is configured as follows:

[0039] The controller acquires first training data based on the first sensor, the first training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle;

[0040] The controller acquires second training data based on the second sensor, the second training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor during a training cycle;

[0041] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter;

[0042] The controller acquires first actual data based on the first sensor, the first actual data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor in a first use cycle;

[0043] The controller inputs the first actual data in the first use cycle into the first neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the first neural network model;

[0044] The controller uses the predicted movement direction parameter, the predicted movement speed parameter and the predicted movement posture parameter as the data of the second sensor in the first use cycle and displays them.

[0045] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0046] Before the controller acquires first training data based on the first sensor, the first training data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle, the method further includes:

[0047] The controller determines the current game scene as the target scene;

[0048] The controller uses the predicted motion direction parameter, the predicted motion speed parameter, and the predicted motion posture parameter as data of the second sensor in the first use cycle and displays them, including:

[0049] Determine whether the current game scene is the target scene. If the current game scene is the target scene, the predicted motion speed parameter and the predicted motion posture parameter are used as the data of the second sensor in the first use cycle and displayed.

[0050] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0051] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the second neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter;

[0052] The controller acquires second actual data based on the second sensor, the second actual data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor in a second use cycle;

[0053] The controller inputs the second actual data in the second use cycle into the second neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the second neural network model;

[0054] The controller uses the predicted movement direction parameter, the predicted movement speed parameter and the predicted movement posture parameter as the data of the first sensor in the second use cycle and displays them.

[0055] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0056] The first usage period does not overlap with the second usage period.

[0057] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0058] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter, including:

[0059] The controller takes the motion direction parameters, motion speed parameters and motion posture parameters in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameters, motion speed parameters and motion posture parameters of the second training data and perform iterative calculations. When the preset conditions are met, the calculation is stopped and the trained first neural network prediction model is output.

[0060] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0061] The controller uses the motion direction parameter, motion speed parameter and motion posture parameter in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter and motion posture parameter of the second training data and perform iterative calculations. When a preset condition is met, the calculation is stopped and the trained first neural network prediction model is output, including:

[0062] Each dimension is normalized so that it lies in the range [0,1].

[0063] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0064] The controller uses the motion direction parameter, motion speed parameter and motion posture parameter in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter and motion posture parameter of the second training data and perform iterative calculations. When a preset condition is met, the calculation is stopped and the trained first neural network prediction model is output, including:

[0065] The controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, and compares the error value t1, the error value t3, and the error value t3 with the preset error value j1, the preset error value j2, and the preset error value j2, respectively, and satisfies:

[0066]

[0067] Where K is a constant.

[0068] In conjunction with the second aspect, in some embodiments, the system is configured as follows:

[0069] The controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, including:

[0070] The error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter satisfy:

[0071] t1=(x0-x v )x v ;

[0072] t2=(y0-y v )y v ;

[0073] t3=(z0-z v ) v ;

[0074] Among them, x0, y0, z0 are the values ​​corresponding to the motion direction parameter, motion speed parameter, and motion posture parameter obtained by the second sensor in the first use period, respectively. v ,y v 、z v They are respectively predicted movement direction parameters, predicted movement speed parameters and predicted movement posture parameters output by the first neural network prediction model according to the first training data.

[0075] A third aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0076] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method proposed in the first aspect of the embodiment of the present invention.

[0077] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the embodiment of the present invention.

[0078] In summary, the above method and device have the following technical effects:

[0079] The embodiment of the present application proposes a wearable sensing interaction system and interaction method based on augmented reality games. In a training cycle, left-hand data and right-hand data are obtained. In a use cycle, only the left-hand sensor can be started and the left-hand data can be obtained. The right-hand data is predicted by combining the neural network prediction model trained with the data in the training cycle with the left-hand data, and the data of both hands are displayed at the same time. Compared with starting the left and right hand sensors at the same time, the total power consumption of the sensor is lower and the battery life is longer, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A flowchart of a wearable sensing interaction method based on augmented reality games proposed in this embodiment. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] The embodiment of the present application proposes a wearable sensing interaction method based on augmented reality games. The method is applicable to a wearable sensing device. The wearable device includes a first motion sensor worn on the left hand, a second motion sensor worn on the left hand, and a controller. The method is applicable to the controller. Figure 1 , the method comprises the following steps:

[0083] S101: The controller obtains first training data based on the first sensor, where the first training data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter obtained by the first sensor during a training cycle.

[0084] It is understandable that the first sensor will continuously acquire and record the motion direction parameter, motion speed parameter and motion posture parameter during the training cycle. During the training cycle, both the first sensor and the second sensor will acquire data.

[0085] S102: The controller acquires second training data based on the second sensor, where the second training data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor during a training cycle.

[0086] It should be noted that, in the present embodiment, the virtual reality device is applied to the game function. Therefore, in different scenes of the game, the user's motion logic may be different. Therefore, in the present embodiment, before the controller obtains the first training data based on the first sensor, and the first training data includes the motion direction parameter, the motion speed parameter, and the motion posture parameter obtained by the first sensor during the training cycle, the method further includes:

[0087] S1021: The controller determines that the current game scene is the target scene.

[0088] It can be understood that this step is to record the current usage scenario as the target scenario to ensure that the subsequent process is in the same scenario and the movement logic is the same.

[0089] S103: The controller inputs the first training data and the second training data within the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameters, motion speed parameters and motion posture parameters in the first training data and the motion direction parameters, motion speed parameters and motion posture parameters of the second training data, and outputs the predicted motion direction parameters, predicted motion speed parameters and predicted motion posture parameters based on the input motion direction parameters, motion speed parameters and motion posture parameters.

[0090] It is understandable that during a person's scene interaction, the movement relationship between his left hand and right hand satisfies a specific logic and is controlled by the human brain. Combined with the simulation process of the neural network, the movement of one hand can be used to predict the movement of the other hand in a specific scene.

[0091] Specifically, in the process of establishing the neural network prediction model, the controller can use the motion direction parameter, motion speed parameter and motion posture parameter in the first training data as three different dimensions, and the three dimensions respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter and motion posture parameter of the second training data and perform iterative calculations. When the preset conditions are met, the calculation is stopped and the trained first neural network prediction model is output. It should be noted that, in order to facilitate calculation, each dimension needs to be normalized so that each dimension is within the range of [0,1].

[0092] It is understandable that normalizing each dimension can make data of different dimensions comparable and avoid different effects of data of different scales on model training. Specifically, let x be the original data of a certain dimension, xmin and xmax be the minimum and maximum values ​​of the dimension respectively, then the normalized value of the dimension is:

[0093] y=(x-xmin) / (xmax-xmin)

[0094] Among them, y is the normalized value, and its value range is [0,1]. This method can preserve the relative size relationship of the data and make the data comparable.

[0095] For example, the neural network training model can use the forward propagation algorithm to take the feature vector as input and calculate the final output value layer by layer. During the calculation process, each neuron uses an activation function to perform a nonlinear transformation on the input value to enhance the expression ability of the model. The final output value usually uses a linear activation function to ensure the continuity of the prediction result.

[0096] Since the training set is composed of the left-hand data and the right-hand data in the training cycle, how to stop the training process can be done by importing the left-hand data into the neural network model, comparing the predicted right-hand data with the actual right-hand data and obtaining the error value. When the error value is less than the preset value, the training can be stopped. As for how to determine the preset value, it can be input by the user or obtained based on big data. The specific process is not limited here.

[0097] Specifically, in this embodiment, the controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, and compares the error value t1, the error value t3, and the error value t3 with the preset error value j1, the preset error value j2, and the preset error value j2, respectively, and satisfies:

[0098]

[0099] Where K is a constant.

[0100] For the determination process of the specific value of the error value, the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter satisfy:

[0101] t1=(x0-x v )x v ;

[0102] t2=(y0-y v )y v ;

[0103] t3=(z0-z v ) v ;

[0104] Among them, x0, y0, z0 are the values ​​corresponding to the motion direction parameter, motion speed parameter, and motion posture parameter obtained by the second sensor in the first use period, respectively. v ,y v 、z vThey are respectively predicted movement direction parameters, predicted movement speed parameters and predicted movement posture parameters output by the first neural network prediction model according to the first training data.

[0105] Of course, in other embodiments, based on different neural network model structures, the error value may be determined by other methods, which are not limited here.

[0106] In order to ensure the average power consumption between the two sensors, based on different usage cycles, the process of using the two sensor data as input sources and prediction sources should be performed alternately, that is, the first usage cycle and the second usage cycle do not overlap. So that the power consumption of the two sensors tends to be the same. Therefore, in contrast to the above steps, the method can also include the following steps:

[0107] S1031: The controller inputs the first training data and the second training data within the training cycle into the first neural network prediction model, and the second neural network prediction model is used to iteratively calculate the motion direction parameters, motion speed parameters and motion posture parameters in the first training data and the motion direction parameters, motion speed parameters and motion posture parameters of the second training data, and outputs the predicted motion direction parameters, predicted motion speed parameters and predicted motion posture parameters based on the input motion direction parameters, motion speed parameters and motion posture parameters.

[0108] The specific implementation process of step S1031 is the same as step S103 and will not be repeated here.

[0109] S104: The controller obtains first actual data based on the first sensor, where the first actual data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter obtained by the first sensor during a first usage cycle.

[0110] It can be understood that in this step, that is, the situation during specific use, the controller obtains first actual data based on the first sensor, that is, the data obtained in the first usage cycle, and the data is used as input data of the trained neural network prediction model.

[0111] Of course, in the second usage cycle, the controller obtains second actual data based on the second sensor, and the second actual data includes the motion direction parameter, motion speed parameter and motion posture parameter obtained by the second sensor in the second usage cycle.

[0112] S105: The controller inputs the first actual data within the first usage cycle into the first neural network training model, and obtains predicted movement direction parameters, predicted movement speed parameters, and predicted movement posture parameters according to the output results of the first neural network model.

[0113] It is understandable that, by using the trained first neural network training model, the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter can be obtained based on the first actual data. At this time, the second sensor can be in a dormant state.

[0114] Of course, the controller inputs the second actual data within the second usage cycle into the second neural network training model, and obtains the predicted movement direction parameters, predicted movement speed parameters and predicted movement posture parameters according to the output results of the second neural network model. At this time, the first controller is in a dormant state.

[0115] S106: The controller displays the predicted movement direction parameter, the predicted movement speed parameter, and the predicted movement posture parameter as the data of the second sensor in the first usage cycle.

[0116] It is understandable that the display of the game screen can be displayed only by the data obtained by one sensor. At this time, the data screen corresponding to the other sensor can be displayed by the predicted data. It should be noted that in order to ensure that it is carried out under the same interactive logic of the same scene, it is also necessary to verify the scene, that is, to determine whether the current game scene is the target scene. If the current game scene is the target scene, the predicted motion speed parameter and the predicted motion posture parameter are used as the data of the second sensor in the first use cycle and displayed.

[0117] Of course, for the second usage cycle, the controller will display the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter as the data of the first sensor in the second usage cycle.

[0118] A wearable sensing interaction method based on augmented reality games is proposed in an embodiment of the present application. In a training cycle, left-hand data and right-hand data are obtained. In a use cycle, only the left-hand sensor can be started and the left-hand data can be obtained. The right-hand data can be predicted by combining the neural network prediction model trained with the data in the training cycle with the left-hand data, and the data of both hands can be displayed at the same time. Compared with starting the left and right hand sensors at the same time, the total power consumption of the sensor is lower and the battery life is longer, thereby improving the user experience.

[0119] Based on the same inventive concept, the embodiment of the present application also proposes a wearable perception interaction system based on augmented reality games, the system includes a first motion sensor worn on the left hand, a second motion sensor worn on the left hand, and a controller, the method is applicable to the controller, and the system is configured as follows:

[0120] The controller acquires first training data based on the first sensor, the first training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle;

[0121] The controller acquires second training data based on the second sensor, the second training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor during a training cycle;

[0122] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter;

[0123] The controller acquires first actual data based on the first sensor, the first actual data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor in a first use cycle;

[0124] The controller inputs the first actual data in the first use cycle into the first neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the first neural network model;

[0125] The controller uses the predicted movement direction parameter, the predicted movement speed parameter and the predicted movement posture parameter as the data of the second sensor in the first use cycle and displays them.

[0126] In some embodiments, the system is configured to:

[0127] Before the controller acquires first training data based on the first sensor, the first training data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle, the method further includes:

[0128] The controller determines the current game scene as the target scene;

[0129] The controller uses the predicted motion direction parameter, the predicted motion speed parameter, and the predicted motion posture parameter as data of the second sensor in the first use cycle and displays them, including:

[0130] Determine whether the current game scene is the target scene. If the current game scene is the target scene, the predicted motion speed parameter and the predicted motion posture parameter are used as the data of the second sensor in the first use cycle and displayed.

[0131] In some embodiments, the system is configured to:

[0132] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the second neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter;

[0133] The controller acquires second actual data based on the second sensor, the second actual data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the second sensor in a second use cycle;

[0134] The controller inputs the second actual data in the second use cycle into the second neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the second neural network model;

[0135] The controller uses the predicted movement direction parameter, the predicted movement speed parameter and the predicted movement posture parameter as the data of the first sensor in the second use cycle and displays them.

[0136] In some embodiments, the system is configured to:

[0137] The first usage period does not overlap with the second usage period.

[0138] In some embodiments, the system is configured to:

[0139] The controller inputs the first training data and the second training data in the training cycle into the first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, motion speed parameter and motion posture parameter in the first training data and the motion direction parameter, motion speed parameter and motion posture parameter of the second training data, and output the predicted motion direction parameter, predicted motion speed parameter and predicted motion posture parameter according to the input motion direction parameter, motion speed parameter and motion posture parameter, including:

[0140] The controller takes the motion direction parameters, motion speed parameters and motion posture parameters in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameters, motion speed parameters and motion posture parameters of the second training data and perform iterative calculations. When the preset conditions are met, the calculation is stopped and the trained first neural network prediction model is output.

[0141] In some embodiments, the system is configured to:

[0142] The controller uses the motion direction parameter, motion speed parameter and motion posture parameter in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter and motion posture parameter of the second training data and perform iterative calculations. When a preset condition is met, the calculation is stopped and the trained first neural network prediction model is output, including:

[0143] Each dimension is normalized so that it lies in the range [0,1].

[0144] In some embodiments, the system is configured to:

[0145] The controller uses the motion direction parameter, motion speed parameter and motion posture parameter in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameter, motion speed parameter and motion posture parameter of the second training data and perform iterative calculations. When a preset condition is met, the calculation is stopped and the trained first neural network prediction model is output, including:

[0146] The controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, and compares the error value t1, the error value t3, and the error value t3 with the preset error value j1, the preset error value j2, and the preset error value j2, respectively, and satisfies:

[0147]

[0148] Where K is a constant.

[0149] In some embodiments, the system is configured to:

[0150] The controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, including:

[0151] The error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter satisfy:

[0152] t1=(x0-x v )x v ;

[0153] t2=(y0-y v )y v ;

[0154] t3=(z0-z v ) v ;

[0155] Among them, x0, y0, z0 are the values ​​corresponding to the motion direction parameter, motion speed parameter, and motion posture parameter obtained by the second sensor in the first use period, respectively. v ,y v 、z v They are respectively predicted movement direction parameters, predicted movement speed parameters and predicted movement posture parameters output by the first neural network prediction model according to the first training data.

[0156] A wearable sensing interaction system based on an augmented reality game proposed in an embodiment of the present application obtains left-hand data and right-hand data in a training cycle. In a use cycle, only the left-hand sensor can be started and the left-hand data can be obtained. The right-hand data can be predicted by combining the neural network prediction model trained with the data in the training cycle with the left-hand data, and the data of both hands can be displayed at the same time. Compared with starting the left and right hand sensors at the same time, the total power consumption of the sensor is lower and the battery life is longer, thereby improving the user experience.

[0157] Based on the same inventive concept, an embodiment of the present application further proposes an electronic device, the electronic device comprising:

[0158] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the wearable perception interaction method based on augmented reality game according to the embodiment of the present application.

[0159] In addition, to achieve the above-mentioned purpose, an embodiment of the present application also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the wearable perception interaction method based on augmented reality games of an embodiment of the present application.

[0160] The following is a detailed introduction to the various components of electronic equipment:

[0161] The processor is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (field programmable gate arrays, FPGA).

[0162] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0163] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0164] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor, or may exist independently and be coupled to the processor through an interface circuit of the electronic device, and the embodiments of the present invention do not specifically limit this.

[0165] A transceiver is used to communicate with a network device or a terminal device.

[0166] Optionally, the transceiver may include a receiver and a transmitter, wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0167] Optionally, the transceiver may be integrated with the processor, or may exist independently and be coupled to the processor via an interface circuit of the router, which is not specifically limited in the embodiment of the present invention.

[0168] In addition, the technical effects of the electronic device can refer to the technical effects of the data transmission method in the above method embodiment, which will not be repeated here.

[0169] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0170] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0171] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0172] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0173] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0174] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0175] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Claims

1. A wearable sensing interaction method based on augmented reality games, characterized in that: The method is applicable to a wearable device, the wearable device comprising a first motion sensor worn on a left hand, a second motion sensor worn on a left hand, and a controller, the method is applicable to the controller, and the method comprises: The controller acquires first training data based on the first sensor, where the first training data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle; The controller acquires second training data based on the second sensor, where the second training data includes the motion direction parameter, the motion speed parameter, and the motion posture parameter acquired by the second sensor during a training cycle; The controller inputs the first training data and the second training data in the training cycle into a first neural network prediction model, the first neural network prediction model is used to iteratively calculate the motion direction parameter, the motion speed parameter and the motion posture parameter in the first training data and the motion direction parameter, the motion speed parameter and the motion posture parameter in the second training data, and outputs a predicted motion direction parameter, a predicted motion speed parameter and a predicted motion posture parameter according to the input motion direction parameter, the motion speed parameter and the motion posture parameter; The controller acquires first actual data based on the first sensor, the first actual data including the motion direction parameter, the motion speed parameter, and the motion posture parameter acquired by the first sensor in a first use cycle; The controller inputs the first actual data in the first usage cycle into the first neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter, and the predicted motion posture parameter according to the output result of the first neural network model; The controller uses the predicted movement direction parameter, the predicted movement speed parameter, and the predicted movement posture parameter as data of the second sensor in the first usage cycle and displays them.

2. A wearable perception interaction method based on augmented reality game according to claim 1, characterized in that: Before the controller acquires first training data based on the first sensor, the first training data including a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle, the method further includes: The controller determines the current game scene as the target scene; The controller uses the predicted motion direction parameter, the predicted motion speed parameter, and the predicted motion posture parameter as data of the second sensor in the first use cycle and displays the parameters, including: Determine whether the current game scene is the target scene. If the current game scene is the target scene, use the predicted motion speed parameter and the predicted motion posture parameter as data of the second sensor in the first usage cycle and display them.

3. A wearable perception interaction method based on augmented reality game according to claim 1, characterized in that: The method further comprises: The controller inputs the first training data and the second training data in the training cycle into a first neural network prediction model, the second neural network prediction model is used to iteratively calculate the motion direction parameter, the motion speed parameter and the motion posture parameter in the first training data and the motion direction parameter, the motion speed parameter and the motion posture parameter in the second training data, and outputs a predicted motion direction parameter, a predicted motion speed parameter and a predicted motion posture parameter according to the input motion direction parameter, the motion speed parameter and the motion posture parameter; The controller acquires second actual data based on the second sensor, the second actual data including the motion direction parameter, the motion speed parameter, and the motion posture parameter acquired by the second sensor in a second use cycle; The controller inputs the second actual data in the second usage cycle into the second neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter according to the output result of the second neural network model; The controller uses the predicted movement direction parameter, the predicted movement speed parameter, and the predicted movement posture parameter as data of the first sensor in the second usage cycle and displays them.

4. A wearable perception interaction method based on augmented reality game according to claim 3, characterized in that: The first usage period does not overlap with the second usage period.

5. The wearable sensing interaction method based on augmented reality game according to claim 1, characterized in that: The controller inputs the first training data and the second training data in the training cycle into a first neural network prediction model, and the first neural network prediction model is used to iteratively calculate the motion direction parameter, the motion speed parameter, and the motion posture parameter in the first training data and the motion direction parameter, the motion speed parameter, and the motion posture parameter in the second training data, and outputs predicted motion direction parameters, predicted motion speed parameters, and predicted motion posture parameters according to the input motion direction parameters, the motion speed parameters, and the motion posture parameters, including: The controller takes the motion direction parameter, the motion speed parameter and the motion posture parameter in the first training data as three different dimensions, which respectively constitute three different feature vectors of the motion direction parameter, the motion speed parameter and the motion posture parameter of the second training data and perform iterative calculations. When a preset condition is met, the calculation is stopped and the trained first neural network prediction model is output.

6. A wearable sensing interaction method based on augmented reality game according to claim 5, characterized in that: The controller uses the motion direction parameter, the motion speed parameter, and the motion posture parameter in the first training data as three different dimensions, and the three dimensions respectively constitute three different feature vectors of the motion direction parameter, the motion speed parameter, and the motion posture parameter of the second training data and performs iterative calculations, and stops calculating and outputting the trained first neural network prediction model when a preset condition is met, including: Each dimension is normalized so that it lies in the range [0,1].

7. A wearable sensing interaction method based on augmented reality game according to claim 6, characterized in that: The controller uses the motion direction parameter, the motion speed parameter, and the motion posture parameter in the first training data as three different dimensions, and the three dimensions respectively constitute three different feature vectors of the motion direction parameter, the motion speed parameter, and the motion posture parameter of the second training data and performs iterative calculations, and stops calculating and outputting the trained first neural network prediction model when a preset condition is met, including: The controller obtains an error value t1 corresponding to the motion direction parameter, an error value t2 corresponding to the motion speed parameter, and an error value t3 corresponding to the motion posture parameter, and compares the error value t1, the error value t3, and the error value t3 with the preset error value j1, the preset error value j2, and the preset error value j2, respectively, and satisfies: Where K is a constant.

8. The wearable sensing interaction method based on augmented reality game according to claim 7, characterized in that: The controller obtains the error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter, including: The error value t1 corresponding to the motion direction parameter, the error value t2 corresponding to the motion speed parameter, and the error value t3 corresponding to the motion posture parameter satisfy: t1=(x0-x v )x v ; t2=(y0-y v )and v ; t3=(z0-z v )With v ; Among them, x0, y0, z0 are the values ​​corresponding to the motion direction parameter, the motion speed parameter and the motion posture parameter acquired by the second sensor in the first use cycle, respectively. v ,y v 、z v They are respectively the predicted motion direction parameter, the predicted motion speed parameter and the predicted motion posture parameter output by the first neural network prediction model according to the first training data.

9. A wearable sensing interaction system based on augmented reality games, characterized in that: The system comprises a first motion sensor worn on the left hand, a second motion sensor worn on the left hand, and a controller, the method is applicable to the controller, and the system is configured as follows: The controller acquires first training data based on the first sensor, where the first training data includes a motion direction parameter, a motion speed parameter, and a motion posture parameter acquired by the first sensor during a training cycle; The controller acquires second training data based on the second sensor, where the second training data includes the motion direction parameter, the motion speed parameter, and the motion posture parameter acquired by the second sensor during a training cycle; The controller inputs the first training data and the second training data in the training cycle into a first neural network prediction model, the first neural network prediction model is used to iteratively calculate the motion direction parameter, the motion speed parameter and the motion posture parameter in the first training data and the motion direction parameter, the motion speed parameter and the motion posture parameter in the second training data, and outputs a predicted motion direction parameter, a predicted motion speed parameter and a predicted motion posture parameter according to the input motion direction parameter, the motion speed parameter and the motion posture parameter; The controller acquires first actual data based on the first sensor, the first actual data including the motion direction parameter, the motion speed parameter, and the motion posture parameter acquired by the first sensor in a first use cycle; The controller inputs the first actual data in the first usage cycle into the first neural network training model, and obtains the predicted motion direction parameter, the predicted motion speed parameter, and the predicted motion posture parameter according to the output result of the first neural network model; The controller uses the predicted movement direction parameter, the predicted movement speed parameter, and the predicted movement posture parameter as data of the second sensor in the first usage cycle and displays them.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; And, a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors so that at least one of the processors can execute the method proposed in any one of claims 1-8.

Images