Intelligent magic wand interaction method and system based on gyroscope gesture recognition

Abstract

The invention relates to the technical field of data interaction recognition, and discloses an intelligent magic wand interaction method and system based on gyroscope gesture recognition, and the method comprises the steps: responding to a starting instruction, collecting a gesture posture, responding to a stopping instruction, and generating first posture data; converting the first gesture data into first gesture data, and matching an operation gesture in a static database based on the first gesture data; if the operation gesture is matched, calling corresponding video content and light content from a content database; otherwise, waiting for the starting instruction again; in response to the operation gesture, collecting a dynamic gesture, generating second gesture data, converting the second gesture data into second gesture data, and matching an action gesture in a dynamic database based on the second gesture data; if the action gesture is matched, the action amplitude of the action gesture is calculated, the playing speed of the video content or the projection range of the light content is adjusted according to positive correlation of the action amplitude, the interaction delay time can be shortened, and the interaction effect is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of data interaction recognition, and in particular to an intelligent magic wand interaction method and system based on gyroscope gesture recognition. BACKGROUND

[0002] With the rapid development of digital entertainment, education and training, virtual simulation and other fields, the demand of users for immersive content experience is increasingly urgent. Traditional human-computer interaction methods such as keyboard, mouse, touch screen and the like are difficult to simulate natural interaction behaviors in real scenes, and cannot meet the requirements of users for real-time, accuracy and immersion in immersive scenes. Therefore, developing an immersive content experience interaction system capable of realizing natural and efficient interaction has become a key to improving user experience and expanding application scenarios of immersive technology, and has important practical significance and market value. At present, the mainstream immersive content experience interaction system mainly consists of an image acquisition module and a display output module. The image acquisition module takes a camera as the core, and the display output module takes a display as the core. Among them, the camera as the core perception component captures the visual information such as the body posture and motion trajectory of the user in real time, converts it into a digital signal and transmits it to the system processing unit. After the processing unit analyzes and identifies the collected image data, corresponding control instructions are generated to drive the display to play dynamic pictures and synchronous sound matched with the user's actions, which is commonly used in VR / AR device matching systems, interactive projection entertainment devices and other application scenarios. However, the immersive content experience interaction system based on the camera needs to perform multiple complex processes such as target detection, feature extraction and action matching on the continuous frame images collected by the camera. The image data is large and the processing algorithm is time-consuming, which leads to significant delay in the generation of interaction instructions, and the user's actions and the feedback pictures of the display are difficult to synchronize, which seriously affects the immersive experience. When there are multiple human targets on site, the images collected by the camera are prone to target occlusion and feature confusion, and the system is difficult to accurately identify and track the actions of the specified interaction user, resulting in a significant decrease in recognition accuracy and even interaction failure, further increasing the delay time of interaction. SUMMARY

[0003] In order to shorten the delay time of the interaction system and improve the interaction effect, the application provides an intelligent magic wand interaction method and system based on gyroscope gesture recognition.

[0004] In the first aspect, the application provides an intelligent magic wand interaction method based on gyroscope gesture recognition, which adopts the following technical scheme: An intelligent magic wand interaction method based on gyroscope gesture recognition, comprising the following steps: In response to the obtained starting instruction, gesture postures are collected in real time based on a preset posture sensor, and first posture data is generated in response to the obtained stopping instruction; the first posture data is converted into first gesture data using a preset first conversion algorithm, and a preset operation gesture is matched in a static database based on the first gesture data; If the operation gesture is matched, video content and light content corresponding to the operation gesture are called from a preset content database, the video content is played through a preset display device, and the light content is projected through a preset light device; otherwise, a matching failure feedback is performed and the starting instruction is re-waited. In response to the matched operation gesture, dynamic postures are collected in real time based on the posture sensor, second posture data is generated, the second posture data is converted into second gesture data using a preset second conversion algorithm, and a preset action gesture is matched in a dynamic database based on the second gesture data. If the action gesture is matched, an action amplitude of the action gesture is calculated, and the playing speed of the video content or the projection range of the light content is adjusted according to the positive correlation of the action amplitude.

[0005] By adopting the above technical solutions, in response to the obtained starting instruction, gesture postures are collected in real time based on a preset posture sensor, and first posture data is generated in response to the obtained stopping instruction; the first posture data is converted into first gesture data using a preset first conversion algorithm, and a preset operation gesture is matched in a static database based on the first gesture data. This process does not need to perform multi-step complex processing such as target detection and feature extraction on continuous frame images collected by a camera, avoids the problems of large image data volume and long processing algorithm time, shortens the time required for generating interactive instructions, keeps the video content and light content synchronized with user actions, and improves the immersive experience. When the operation gesture is matched, video content and light content corresponding to the operation gesture are called from a preset content database, the video content is played through a preset display device, and the light content is projected through a preset light device, to realize the basic presentation of immersive content experience interaction. At the same time, in response to the matched operation gesture, dynamic postures are collected in real time based on the posture sensor to generate second posture data, the second posture data is converted into second gesture data using a preset second conversion algorithm, a preset action gesture is matched in a dynamic database based on the second gesture data, the action amplitude of the action gesture is calculated, and the playing speed of the video content or the projection range of the light content is adjusted according to the positive correlation of the action amplitude, to make the correlation between operation and feedback in the interactive process stronger, and improve the coherence and adaptability of content interaction.

[0006] Further, the first posture data includes a plurality of first posture values, and the first conversion algorithm includes the following steps: Based on the acquisition of abnormal threshold range, the difference between adjacent first attitude values is calculated as attitude difference value, if the attitude difference value is located in the abnormal threshold range, the average value of the two first attitude values corresponding to the attitude difference value is used to replace the two first attitude values; Based on the acquisition cycle of the gesture attitude, the filter window length corresponding to the acquisition cycle is obtained; Based on the filter window length, the first attitude data is window filtered; According to the NNOM model, the first gesture data with content of graphics is identified from the first attitude data; or, the action direction is identified from the first attitude data, the direction vector is extracted according to the action direction, the direction vector sequence is obtained by sorting the direction vector based on time sequence, and the first gesture data is matched from the preset graphic database according to the direction vector sequence.

[0007] By adopting the above technical solutions, the first attitude data is abnormally processed and window filtered, the data stability is maintained, the NNOM model or the direction vector sequence is matched with the first gesture data, the operation gesture matching process is smoother, the timeliness of interactive feedback is maintained, the coherence of immersive content experience is strengthened, and the association between gesture and feedback is more consistent with the expectation.

[0008] Further, in the step of matching the first gesture data with the preset operation gesture in the static database, the following sub-steps are further included: According to the first gesture data, the gesture similarity value is calculated by comparing the elements in the static database one by one, and the gesture confidence value is calculated according to the gesture similarity value and the preset reference similarity value; the maximum gesture confidence value is selected as the final gesture confidence value; If the final gesture confidence value is greater than the preset reference confidence value, the corresponding element is taken as the matched operation gesture and the operation gesture is outputted; The ratio of the final gesture confidence value to the reference confidence value is calculated as the confidence ratio value, and the filter window length is adjusted according to the positive correlation of the confidence ratio value.

[0009] By adopting the above technical solutions, the gesture similarity value and the gesture confidence value are calculated, the maximum gesture confidence value is screened, the operation gesture matching accuracy is improved, and the mis-matching situation is reduced; the filter window length is adjusted according to the confidence ratio value, the adaptability of data processing and gesture recognition is maintained, the matching of the first gesture data and the operation gesture is more stable, the gesture basis is more consistent for subsequent calling of video and light content, and the immersive interactive experience is improved.

[0010] Further, the second attitude data includes a plurality of second attitude values; the second conversion algorithm includes the following steps: According to the preset data length, the plurality of second attitude values are divided into a plurality of attitude groups in time sequence; Calculate the average of all second posture values in each posture group, and subtract the average from all second posture values in each posture group to obtain second posture dynamic values, and update the second posture data using the second posture dynamic values; Calculate the one-way fluctuation interval of the second posture data, which is the range interval of a plurality of continuous data with the same trend; Calculate the interval number and interval interval of the one-way fluctuation interval, and calculate the interval average according to the interval interval, wherein the interval interval is the time interval between the one-way fluctuation intervals; Calculate the number calculation value according to the interval number and the preset interval comparison number, and calculate the interval calculation value according to the interval interval and the preset interval comparison interval; Set the second gesture data, write the number calculation value and the interval calculation value into the second gesture data, and output the second gesture data.

[0011] By adopting the above technical scheme, the number calculation value and the interval calculation value are obtained by processing the second posture data and written into the second gesture data, which is beneficial to make the second gesture data fit the actual situation of dynamic posture, maintain the integrity of the second gesture data, provide adaptive basis for subsequent matching of action gestures in the dynamic database, reduce data deviation interference, and improve the smoothness of interactive feedback.

[0012] Further, in the step of matching the preset action gesture based on the second gesture data in the dynamic database, the following sub-steps are further included: Extract the number calculation value and the interval calculation value from the second gesture data; If the number calculation value is less than the preset number setting value or the interval calculation value is greater than the preset interval setting value, the matching fails; otherwise, a matching calculation value is calculated by weighted average according to the number calculation value and the interval calculation value; Calculate the gesture difference value by difference calculation between the matching calculation value and the feature value corresponding to each element in the dynamic database; filter out the smallest gesture difference value, and take the corresponding element as the matched action gesture and output the action gesture; Adjust the weighted value of the number calculation value according to the positive correlation of the gesture difference value, and adjust the weighted value of the interval calculation value according to the negative correlation.

[0013] By adopting the above technical scheme, the matching calculation value is calculated by extracting the value, and the smallest gesture difference value is filtered out, which is beneficial to improve the matching accuracy of the action gesture and reduce the mismatch; the weighted value is adjusted according to the gesture difference value, which can maintain the adaptability of subsequent matching and make the action gesture matching more stable.

[0014] Further, the method further includes the following steps: Obtain a gesture volume library corresponding to the operation gesture and the action gesture; According to the operation gesture, the operation basic value is matched, and according to the action gesture, the action basic value is matched. The execution duration of the operation gesture is calculated, and the duration of the action gesture is calculated. According to the execution duration and the duration, the execution weight value and the duration weight value are calculated. According to the operation basic value and the execution weight value, the operation volume value is calculated, according to the action basic value and the duration weight value, the action volume value is calculated, according to the operation volume value and the action volume value, the final volume value is calculated, and according to the final volume value, the volume of the played video content is controlled.

[0015] By adopting the above technical scheme, the operation basic value and the action basic value are matched relying on the gesture volume library, the final volume value is obtained by combining the duration calculation weight value, which is beneficial to make the volume control fit the gesture operation; meanwhile, the relevance between the volume adjustment and the gesture execution and duration is maintained, the adaptability of the volume control in the interaction is improved, and the dynamically adapted volume output is provided for the video content playing.

[0016] Further, the method further comprises the following steps: According to the matching calculation value and the characteristic value of the corresponding element, the matching ratio value is calculated. The ratio of the confidence ratio value and the matching ratio value is the adjustment ratio value, and the operation basic value is adjusted in positive correlation according to the adjustment ratio value, and the action basic value is adjusted in negative correlation.

[0017] By adopting the above technical scheme, the basic value is adjusted through the adjustment ratio value, which is beneficial to improve the adaptability of the basic value and the gesture, maintain the dynamic nature of the volume calculation, make the final volume control more adaptive to the interactive demand, and enhance the matching degree of the gesture and the volume adjustment.

[0018] Further, the method further comprises the following steps: The camera module of the alignment posture sensor is set, and based on the starting instruction, the camera module collects the action trajectory of the posture sensor to generate trajectory data; According to the trajectory data, the operation trajectory is matched from the preset trajectory database; If the operation trajectory is matched, the stop instruction corresponding to the starting instruction is obtained, and the first operation gesture after the stop instruction is obtained. The shape matching value of the operation trajectory and the operation gesture is calculated, if the shape matching value is less than the preset set matching value, the trajectory recognition is prompted not to correspond, the shape adjustment value is calculated according to the shape matching value and the set matching value, and the reference confidence value is adjusted in negative correlation according to the shape adjustment value.

[0019] By adopting the technical scheme, the trajectory data is collected by the camera module and the operation trajectory is matched, which is beneficial to the corresponding matching of the operation trajectory and the operation gesture, reduces the situation that the trajectory recognition does not correspond, and adjusts the reference confidence value according to the shape matching value, so that the adaptability of subsequent gesture matching can be maintained, the overall matching precision is dynamically improved, and the matching stability in the interaction process is enhanced.

[0020] Further, the method further comprises the following steps: The acceleration data in the attitude sensor is acquired in real time, the shooting angle of the camera module is adjusted according to the acceleration data, the angle of the camera module corresponds to the attitude of the attitude sensor, and the acceleration data is acquired in real time. The average value of the nearest plurality of shape matching values is calculated to obtain a shape average value, and the angle adjustment step and the shooting frame rate of the camera module are adjusted according to the negative correlation of the shape average value.

[0021] By adopting the technical scheme, the camera angle is adjusted according to the acceleration data, which is beneficial to keeping the camera angle corresponding to the angle of the attitude sensor; the shape average value adjusts the step and the frame rate, which is beneficial to dynamic adaptation, improves the trajectory matching effect, and also reduces the amount of shooting data.

[0022] In a second aspect, the application provides an intelligent magic wand interaction system based on gyroscope gesture recognition, which adopts the following technical scheme: An intelligent magic wand interaction system based on gyroscope gesture recognition, comprising a processor, wherein the processor executes the steps of the intelligent magic wand interaction method based on gyroscope gesture recognition according to any one of the above. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 It is a method flowchart of the intelligent magic wand interaction method based on gyroscope gesture recognition.

[0024] Figure 2 It is a logical flowchart around the NNOM model algorithm.

[0025] Figure 3 It is a processing flowchart of projection.

[0026] Figure 4 It is a business logic flowchart. DETAILED DESCRIPTION

[0027] The embodiments of the application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0028] The embodiment of the application discloses a smart magic wand interaction method based on gyroscope gesture recognition. The hardware device of the smart magic wand comprises an appearance structure assembly and an internal function module. The appearance structure assembly comprises a top end assembly, a wand body assembly and a bottom end assembly. The top end assembly is provided with a fluorescent column, the inside of which is integrated with a 940nm infrared LED lamp, which is used as an infrared recognition light source carrier to form an infrared camera assembly. The wand body assembly is sequentially provided with a white light transmission area, a touch area, a PCB double-sided panel, a main control panel mounting position and a charging battery along an axial direction. The surface of the white light transmission area is sprayed with a PT:4271C coating, which is beneficial to guarantee the light transmission performance. The touch area is provided with a constant light bead at a corresponding position, which is used as a touch interaction identifier. The PCB double-sided panel is provided with 13 RGB LED single-sided panels, which are used for wand body light effect output. The main control panel mounting position is used for mounting a main control chip. The charging battery adopts a 14500 model to provide hardware power supply. The wand body shell is sprayed with a PT:7596C coating, the bottom color of which is black matte, and the local area is processed by twice anti-dulling paint treatment and imitation pollution process; the lower part of the wand body is provided with one RGB LED lamp bead. The bottom end assembly adopts a resin crystal component, which is used as a structure tail and an appearance decoration piece.

[0029] The internal function module comprises an MCU hardware control core, a Bluetooth 4.2 transceiver module, a 6-axis IMU module, an RGB LED patch lamp group, a 940nm infrared LED lamp, a vibration module and a touch module, all of which are electrically connected with the main control panel and are uniformly dispatched by the MCU. The working process of the hardware is realized by the cooperation of the MCU and each module. When starting, the starting is triggered by a vibration action, after starting, the wand body RGB LED executes a flowing light effect, the top end RGB LED executes a breathing light effect, and an initial feedback is generated. After starting for 3 seconds, the Bluetooth 4.2 module enters a constant connection state, which is used for establishing a link with a matching magic mirror device, and the 940nm infrared LED lamp is always on, which provides an infrared recognition light source for the matching device to realize communication and recognition preparation. In the process of hardware execution of body gesture recognition, when the wand body is waved, the 6-axis IMU module starts data collection, and gestures such as flashing, swinging, turning and love can be recognized. The self-detection process of the hardware is as follows: first, the pressing state of the touch area is detected, the speed threshold is calculated after the action is paused for more than 100ms, and the IMU collection is started; the waving time length needs to be less than or equal to 30ms and meet the effective sample amount, so as to enter the next program; otherwise, it is determined as an invalid gesture, and no feedback is executed. In the idle state, the top end RGB LED presents a breathing light effect, and the brightness of the light effect is adaptively adjusted according to the signal strength; the top end LED is blue during gesture collection; after the effective collection is completed, the wand body LED plays a flowing light effect, and the top end LED is extinguished after the end; when the effective gesture reasoning is completed, the vibration module executes vibration to realize sound and light and tactile feedback. The effective gesture data is sent to the magic mirror device through Bluetooth, the magic mirror device is a matching display device, the magic mirror device returns the result after matching, and corresponding light effects are executed according to the result. When the battery power is lower than the threshold, the MCU controls automatic shutdown to realize interaction and low power protection.

[0030] Referring to Figure 1 , the method comprises the following steps: in response to the obtained start instruction, collecting gesture posture in real time based on a preset posture sensor. For example, the data collection of the gesture posture uses a single-chip microcomputer and a posture sensor arranged on the smart wand. The single-chip microcomputer can use STM32, and the posture sensor can use a magnetic field sensor and MPU6050 or BMI160. The posture sensor is connected with the MCU to collect at a fixed frequency. The accelerometer data includes acc_x, acc_y, and acc_z. The gyroscope data includes gyro_x, gyro_y, and gyro_z. The magnetic field sensor data includes mag_x, mag_y, and mag_z. The posture data after the fusion of the accelerometer and the gyroscope can also be directly obtained through the DMP motion calculation unit built in the posture sensor, and then the data can be further fused with the magnetic field sensor data. An infrared camera assembly can also be formed through an infrared LED lamp and an infrared camera to identify the gesture posture through infrared.

[0031] In response to the obtained stop instruction, first posture data is generated, and the first posture data includes a plurality of first posture values. The first posture data is converted into first gesture data using a preset first conversion algorithm. The first conversion algorithm can obtain an abnormal threshold range, calculate the difference between adjacent first posture values as a posture difference value, and if the posture difference value is within the abnormal threshold range, replace the two first posture values with the average value of the two first posture values corresponding to the posture difference value, so as to realize the identification and replacement of abnormal values in the first posture data. Based on the collection period of the collected gesture posture, a filter window length corresponding to the collection period is obtained, and the first posture data is window filtered based on the filter window length, so as to realize the smoothing of the fluctuation in the first posture data.

[0032] Suppose the smart wand collects a "static stay" stop gesture, and the posture sensor (such as MPU6050) collects acceleration data at a period of 10 ms / time. After triggering the stop instruction, the data after removing the g influence is: [1.1, 1.2, 1.3, 3.2, 1.4, 1.3], wherein 3.2 is an abnormal value caused by hand instantaneous jitter. According to the first conversion algorithm, the abnormal threshold range ±0.5m / s 2, the adjacent pose difference is calculated: 1.2-1.1=0.1 (within the threshold, unchanged), 1.3-1.2=0.1 (unchanged), 3.2-1.3=1.9 (exceeds the threshold), so the average value 2.25 of 1.3 and 3.2 is used to replace the two values, and the data becomes [1.1, 1.2, 2.25, 2.25, 1.4, 1.3]; then the next group of difference values is calculated 2.25-1.4=0.85 (still exceeds the threshold), and the average value 1.825 of 2.25 and 1.4 is used to replace, and the final modified data is [1.1, 1.2, 2.25, 1.825, 1.825, 1.3]. Then according to the 10ms acquisition period, the filter window length is determined to be 4, and the sliding average filter is performed on the modified data: the average value of the first four data (1.1+1.2+2.25+1.825) / 4≈1.59, and the subsequent window is calculated in turn, and the smoothed first gesture data [1.1, 1.2, 1.59, 1.69, 1.64, 1.3] is obtained, which eliminates abnormal values and small fluctuations, and is more consistent with the gesture characteristics of "static stay".

[0033] In this embodiment, the first gesture data with the content of graphics is identified from the first pose data according to the NNOM model. As shown in FIG. 6, the first gesture data is obtained by the sensor data acquisition, data preprocessing, feature extraction, NNOM model inference, gesture classification, confidence detection, and special effect triggering or gesture ignoring. Figure 2 , the algorithm logic around the NNOM model is as follows: first, sensor data acquisition is performed, then data preprocessing is performed, feature extraction is performed, the NNOM model is used for inference, gestures are classified, and confidence detection is performed. If it exceeds the threshold, a special effect is triggered, otherwise the gesture is ignored.

[0034] After data cleaning, data alignment and segmentation are performed first, and each gesture sample is segmented into a fixed length time window, such as 50 time steps, to ensure that all sample lengths are consistent. Then feature extraction is performed, and the original time series data is directly used to train the model, such as using LSTM to calculate statistical features, and converting the time series data into a fixed-dimension feature vector: mean, standard deviation, extreme value, peak value, correlation, etc. For example, for a 50-step x 9-axis gesture sample, we can calculate 5 statistical features for each axis, and finally obtain a 45-dimensional feature vector. Keras model training is performed, and since the input is a feature vector or a time series sequence, different model architectures can be selected. Model architecture A: use multi-layer perceptron MLP- suitable for feature vector-based input; model architecture B: use one-dimensional convolutional network 1D-CNN- suitable for original time series data. Similar to the image classification process, the NNOM model is used to convert the trained Keras model for real-time inference, wherein not all features are beneficial to classification in feature selection, and feature importance analysis such as random forest can be used to screen the most important features, reducing the input dimension and calculation amount. The calibration data used for NNOM model conversion can represent the real input distribution. The feature extraction algorithm implemented on the MCU uses integer operations instead of floating point operations.

[0035] In another embodiment, the action direction is identified from the first posture data, a direction vector is extracted according to the action direction, the direction vector is sorted based on the time sequence to obtain a direction vector sequence, and the first gesture data is matched from a preset graph database according to the direction vector sequence. The first posture data is subjected to abnormal processing and window filtering to maintain data stability, facilitate the matching of the first gesture data by the NNOM model or the direction vector sequence, make the operation gesture matching process smoother, maintain the timeliness of interactive feedback, facilitate the coherence of the immersive content experience, and make the association between the gesture and the feedback more consistent with the expectation.

[0036] The following calculations can be performed in the gateway, background or host computer, which runs in the field PC. In this embodiment, the gateway is used as the gesture judgment core, and the host computer is used as the core for controlling the field device. The first gesture data is matched with the preset operation gesture in the static database. The operation gesture in the static database can be defined as: swinging a bat (swinging left and right), drawing a figure (changing the direction of the polyline), turning (rotating around an axis), etc. According to the first gesture data, the similarity value is calculated with each element in the static database to obtain the gesture similarity value. The gesture similarity value is calculated with the preset reference similarity value to obtain the gesture confidence value. The largest gesture confidence value is selected as the final gesture confidence value. If the final gesture confidence value is greater than the preset reference confidence value, the corresponding element is taken as the matched operation gesture and the operation gesture is output. The ratio of the final gesture confidence value to the reference confidence value is the confidence ratio value, and the filter window length is adjusted according to the positive correlation of the confidence ratio value. By calculating the gesture similarity value and the gesture confidence value, and screening the largest gesture confidence value, the operation gesture matching accuracy is improved, and the mis-matching situation is reduced. According to the confidence ratio value, the filter window length is adjusted, which can maintain the adaptability of data processing and gesture recognition, make the matching of the first gesture data and the operation gesture more stable, provide more suitable gesture basis for subsequent calling of video and light content, and improve the immersive interactive experience. The gateway is the key node connecting the magic wand, the host computer and the matching display device, and is the core control channel of the intelligent magic wand. It can switch between multiple working modes according to different interaction requirements. If the working mode is gesture gateway, the magic wand is first notified that it is currently in the gateway type, and then the received magic wand gesture data is verified. Meanwhile, the magic wand's Bluetooth transmission power can be adjusted through the USB serial port, and 10 adjustable gears can be set. When working, it compares the magic wand gesture data with the static database, and if the matching is successful, it feeds back the confirmation instruction and the corresponding operation gesture to the magic wand, and if the matching is unsuccessful, it does not respond, and at the same time, it reports the gesture verification result and the corresponding MAC address of the magic wand to the host computer, which helps to ensure the accuracy of gesture interaction. Specifically, the magic wand enters the gateway range and receives the gateway type signal. The gateway only listens to the corresponding gestures of the magic wand, and other gestures are invalid. When the gateway receives the successful and matched magic wand gesture, it reports to the PC and sends the magic wand feedback instruction. The magic wand responds to the feedback instruction to control the LED lamp bead which has a mapping relationship with the corresponding gesture to be always on.

[0037] Specifically, the magic wand gesture is completed, a request is initiated to the gateway, the request content contains the magic wand MAC, the gateway replies according to the received MAC address, the reply content contains the magic wand MAC, the magic wand checks whether it is the same as the MAC address, if yes, the magic wand sends the gesture result to the gateway, the gesture result includes the magic wand MAC and the gesture result in the request, the gateway receives it and judges whether it is the predetermined gesture, if yes, it replies to the magic wand, and whether it is correct or not is reported to the host computer, which executes and determines whether to start the device in the field.

[0038] If there are multiple smart wands, the working mode is to pair the gateway, first notify the wand that it is currently in the gateway type, synchronize the current pairing mode state to the wand, identify the 5-second reference time window, which can be relaxed to 10 seconds at most, and determine whether multiple wands have completed communication connection within the reference time window; when it is detected that the conditions for multiple wand interaction success within the set time are met, the pairing configuration process is automatically started, and the communication time upper limit can also be set through the USB serial port, and finally all the wand MAC addresses participating in this pairing are reported to the upper computer, providing support for multiple wand collaborative interaction. Specifically, the wand enters the gateway range and receives the gateway type signal, the gateway identifies whether three wands are in the interactive state at the same time within the reference time window, and if the identification is successful, the PC and the wand are reported, and the wand is bright The top RGB LED is always on.

[0039] If it needs to be reset, the working mode is to reset the gateway, first notify the wand that it is currently in the gateway type, receive the upper computer instruction and send the reset signal to the wand; after receiving the execution instruction, it will issue a reset instruction, and after receiving it, the wand will switch the corresponding light effect feedback according to its own state, and at the same time, the gateway will report the MAC address of the target wand, realizing the fast reset of the wand state.

[0040] If the operation gesture is matched, the video content and light content corresponding to the operation gesture are called from the preset content database, the video content is played through the preset display device, and the light content is projected through the preset light device; otherwise, a matching failure feedback is performed and a start instruction is re-waited. The display device uses a projector and a curtain, the light device uses laser projection and the light effect of the wand itself, and the matching failure feedback provides a tactile feedback for the built-in vibration motor. The single-chip microcomputer is connected with a wireless module and is wirelessly connected with the background, is used for calling video content in the background and displaying it on the curtain through the projector, also calling linear light effects and displaying them on the ground or wall through laser projection, and controlling the lamp beads or lamp strips on the wand to display light effects. Assuming that the user holds the smart wand to complete the "draw a triangle" operation gesture, after the system successfully matches the gesture in the static database, the single-chip microcomputer establishes a connection with the background through the wireless module, and calls the preset "magic triangle blooming" theme video and blue-purple laser light effect from the content database. The background immediately instructs the projector to project this video onto the curtain, and a rotating triangular magic array appears in the picture; at the same time, the laser projection projects a flickering linear triangular light effect on the ground, and the RGB lamp beads of the wand also synchronously light up the blue-purple flowing light, forming an immersive interactive effect. If the user does the "random swing" gesture without a preset corresponding relationship, after the system fails to match, the vibration motor built in the wand will give a short tactile feedback, prompting that the gesture is not recognized, and then the system re-enters the waiting state, waiting for the user to trigger the start instruction again and make an effective operation gesture.

[0041] As Figure 3As shown, the processing flow of the projector is designed around a "standby-trigger-feedback-reset" closed loop, always maintaining high efficiency response. In daily state, the projector is in standby mode, playing the "fantasy magic array standby" video in a loop; the light effect of light blue color in the picture slowly rotates, creating an immersive atmosphere, while the background system listens to the gesture events of the smart wand through the wireless module in real time. If the user makes random shaking without preset corresponding relationship, or the gesture is blurred and the recognition fails, the projector will continue to maintain standby video playback, and the listening function will not be interrupted; once the user completes the effective operation gesture of "drawing a circle" and matches successfully, the projector will immediately stop the standby video, seamlessly switch to the "crystal ball blooming" special effect video, and the picture will explode with a circular magic array and starlight scattered, and after 15 seconds of special effect video playback, it will automatically return to standby state, waiting for the next user trigger.

[0042] In response to the matching successful operation gesture of "drawing a circle", the BMI160 posture sensor built-in the wand immediately starts the dynamic acquisition mode, with a high-frequency acquisition period of 10ms / second, to capture the user's subsequent dynamic posture; for example, after the user draws a circle, the wand is quickly waved left and right, and the sensor continuously acquires 12 groups of acceleration and angular velocity fusion data, generating second posture data containing 12 second posture values: [1.8, 2.1, 2.4, 2.2, 1.9, 1.6, 1.3, 1.5, 1.7, 2.0, 2.3, 2.1]. According to the preset data length 6 (i.e. every 6 consecutive values is a group), the system divides the second posture values into two posture groups in time sequence: the first group [1.8, 2.1, 2.4, 2.2, 1.9, 1.6], and the second group [1.3, 1.5, 1.7, 2.0, 2.3, 2.1]. Then the average value of all second posture values in each group is calculated, the first group average value is (1.8+2.1+2.4+2.2+1.9+1.6) / 6=2.0, and the second group average value is (1.3+1.5+1.7+2.0+2.3+2.1) / 6=1.8. Subtract the corresponding average value from each second posture value in each group to obtain the second posture dynamic value (first group: [-0.2, 0.1, 0.4, 0.2, -0.1, -0.4]; second group: [-0.5, -0.3, -0.1, 0.2, 0.5, 0.3]), and update the original second posture data with these dynamic values to highlight the core features of posture changes.

[0043] Based on the updated second attitude data, the system further analyzes the motion trend and identifies the one-way fluctuation interval; that is, the data range that is continuous and consistent in motion trend (upward or downward). After calculation, three one-way fluctuation intervals are finally determined: [-0.2, 0.1, 0.4, 0.2] (upward trend), [-0.1, -0.4, -0.5, -0.3, -0.1] (downward trend), and [0.2, 0.5, 0.3] (upward trend). The number of intervals is 3, and the interval interval (the time interval between two adjacent one-way fluctuation intervals) is 20 ms (corresponding to 2 collection periods), 20 ms, and the average interval is (20+20) / 2=20 ms. Then, based on the preset interval comparison number 4 and interval comparison interval 30 ms, the number calculation value=number of intervals / interval comparison number=3 / 4=0.75 and the interval calculation value=interval average / interval comparison interval=20 / 30≈0.67 are calculated by normalization. These two core parameters are written into the second gesture data and output, providing accurate basis for subsequent motion matching.

[0044] The system extracts the number calculation value of 0.75 and the interval calculation value of 0.67 from the output second gesture data, and makes a preliminary judgment based on the preset rules: the preset number setting value is 0.5 (below this value, the motion trend is not obvious), and the interval setting value is 1.0 (above this value, the motion continuity is poor). Since 0.75≥0.5 and 0.67≤1.0, it is determined that the matching conditions are met. Then, according to the preset weighting ratio (number calculation value weight 0.6, interval calculation value weight 0.4), the matching calculation value=0.75×0.6+0.67×0.4≈0.718 is calculated. The matching calculation value is compared with the preset action gesture characteristic values in the dynamic database one by one; the dynamic database includes "fast left and right waving" (characteristic value 0.7), "slow up and down swinging" (characteristic value 0.3), "uniform speed drawing circle" (characteristic value 0.5), etc. The gesture difference values are calculated as 0.018, 0.418, and 0.218, respectively, and the minimum difference value 0.018 corresponding to "fast left and right waving" is selected as the matching successful action gesture. At the same time, according to the rule of "positive correlation adjustment of number calculation value weight value and negative correlation adjustment of interval calculation value weight value", the weight value of number calculation value is adjusted from 0.6 to 0.62, and the weight value of interval calculation value is adjusted from 0.4 to 0.38, which optimizes the parameter adaptability of subsequent matching.

[0045] After matching the "quick left-right swing" motion gesture, the system converts the amplitude of the motion to 3.5 (value range 0-5, the larger the value, the more intense the motion) based on the peak difference of the second posture dynamic value. According to the rule that "motion amplitude is positively correlated with adjustment effect", the playback speed of the "crystal ball blooming" special effect video is increased from 1 times to 1.9 times, so that the picture rhythm and motion intensity echo each other; at the same time, the projection range of the laser projection is expanded from the initial 2.5 meters radius to 4.5 meters, covering a larger interactive space. At the same time, the system calls the preset gesture volume library, and the operation basis value of the "draw a circle" operation gesture is 65 (volume scale 0-100), and the motion basis value of the "quick left-right swing" motion gesture is 75; the execution time of the operation gesture is 2.2 seconds, and the duration of the motion gesture is 1.8 seconds, and the weight value is obtained after normalization processing (execution weight value = 2.2 / (2.2+1.8) = 0.55, duration weight value = 1.8 / (2.2+1.8) = 0.45). Combined with the matching calculation value 0.718 and the corresponding characteristic value 0.7 calculated before, the matching ratio ≈1.026, and the confidence ratio of the operation gesture matching is 1.15, the adjustment ratio = 1.15 / 1.026 ≈1.121, and the operation basis value is adjusted according to the rule of positive correlation (65x1.121 ≈72.87), and the motion basis value is adjusted according to the rule of negative correlation (75÷1.121 ≈66.91). Finally, the operation volume value = 72.87x0.55 ≈40.08, the motion volume value = 66.91x0.45 ≈30.11, and the final volume value ≈70.19 is obtained by superposition, and the system automatically adjusts the video volume to 70 scales, realizing the deep adaptation of volume and gesture operation, dynamic motion.

[0046] The core advantage of the whole interaction process is "no redundant data processing": after responding to the start instruction, the posture sensor directly collects the gesture posture, generates the first posture data and converts it into the first gesture data, without the need for complex operations such as target detection and feature extraction of continuous frame images as in the traditional camera scheme. This design avoids the problems of large image data and time-consuming algorithms, shortens the delay of generating interaction instructions from more than 500ms in traditional schemes to within 100ms, ensures that video playback, light projection, and volume adjustment are synchronized with user actions in real time, and greatly improves the immersive experience. When the operation gesture is matched, the system quickly calls the corresponding video and light content to complete the basic presentation of immersive interaction; subsequently, through dynamic posture collection, data conversion, motion matching and multi-dimensional adjustment, the correlation between operation and feedback is stronger; for example, the larger the motion amplitude, the faster the video, the wider the light range, and the higher the volume, forming a closed-loop adaptation of "motion-feedback", which significantly improves the coherence and adaptability of content interaction.

[0047] The overall business logic is as follows Figure 4As shown, the visitor can draw gestures by holding the magic wand device, the single-chip microcomputer on the magic wand device collects sensor data of the posture sensor, and sends the sensor data to the main system in the background, and the main system performs gesture recognition processing. If the recognition is successful, the media system containing the projector and the laser projection triggers special effect video and light effect, and shows multimedia special effects to the visitor. If the recognition fails, send a vibration feedback, and play a failure prompt through the media system.

[0048] To provide negative feedback on the results of gesture recognition and further improve interaction accuracy, a camera module for aligning the magic wand posture sensor (such as BMI160) can also be specially set. If the application scene is dark (such as immersive escape room), a 940nm infrared camera component can be selected to avoid environmental light interference. If in a bright interactive exhibition hall, a high-definition visible light camera component is used to ensure trajectory capture clarity. After adding the camera module, the working mode of the system gateway is also correspondingly added with a "camera gateway": when the user enters the interactive area that needs trajectory verification, the camera gateway will first send the "current camera gateway mode" instruction to the magic wand through Bluetooth, and automatically interface the external camera module with the background trajectory recognition program. The camera gateway itself does not participate in complex data operation, and only forwards the unique MAC address (such as "AA:BB:CC:DD:EE:FF") of the magic wand to the background, providing stable data transmission support for "trajectory + gesture" dual verification interactive scenes. The entire gateway system can be flexibly switched between gesture gateway, pairing gateway, reset gateway and camera gateway: when single-person operation, switch to gesture gateway to ensure accurate recognition, when multiple people interact, switch to pairing gateway to realize multi-magic wand linkage, when device is abnormal, use reset gateway to quickly restore state, and when trajectory verification is needed, switch to camera gateway to expand interaction dimension. The intelligent magic wand system can adapt to single-person immersive experience and meet different scene needs such as multi-person cooperation and trajectory interaction, greatly improving the overall compatibility and flexibility.

[0049] Based on the user triggered start instruction (such as long press wand touch area for 2 seconds), the camera module is immediately started and aligned with the posture sensor, and the motion trajectory of the user waving the wand is collected in real time; for example, the user draws a "five-point star" in the air, and the camera module will continuously capture the key points of the trajectory to generate trajectory data containing coordinates and timing information. The system matches this trajectory data with the preset trajectory database (containing standard trajectory templates such as "five-point star", "triangle", "circle", etc.), and if the "five-point star" operation trajectory is successfully matched, the corresponding stop instruction (user releases the touch area) is automatically obtained, and the first operation gesture after the stop instruction (such as the user's posture after drawing the five-point star and keeping the wand still) is extracted. Then the system calculates the shape matching value of the operation trajectory (five-point star) and the operation gesture (the shape feature corresponding to the still posture), assuming that the preset set matching value is 0.8 (0-1 interval, the closer to 1, the more matched), if the actual calculated shape matching value is 0.5 (because the user's five-point star is not regular enough, and the gesture feature deviates greatly), the "trajectory recognition does not correspond" is prompted through the built-in light feedback of the wand (red light flashes 3 times quickly); at the same time, according to the shape matching value and the set matching value, the shape adjustment value = 0.5 / 0.8 = 0.625 is calculated, and according to the "negative correlation adjustment" rule, the original reference confidence value 0.7 is lowered to 0.7x0.625 = 0.4375, so that the subsequent gesture matching judgment standard is more strict, and the false matching is reduced.

[0050] The system will obtain the acceleration data in the posture sensor in real time; for example, the user tilts the wand from the horizontal state to 30° upward, and the accelerometer Z-axis data changes from 1.0 m / s 2 to 3.2 m / s 2 , the system automatically adjusts the shooting angle of the camera module according to this data, and the camera lens is turned up by 30° synchronously, so that the posture of the camera lens always corresponds to the posture of the posture sensor, avoiding the trajectory capture deviation caused by the movement of the wand. At the same time, the system will record the shape matching values of the last 5 times (such as 0.5, 0.55, 0.6, 0.58, 0.62), and calculate the shape average value = (0.5+0.55+0.6+0.58+0.62) / 5 = 0.57. According to the "shape average value negative correlation adjustment" rule, because the shape average value is low (indicating that the trajectory matching accuracy needs to be improved), the angle adjustment step of the camera module is expanded from 0.5° / time to 1° / time, and the frame rate is increased from 30 frames / second to 60 frames / second, so that the camera module can adapt to the change of the wand posture faster and capture the trajectory details more clearly; if the subsequent shape average value is improved to 0.85 (the matching accuracy is improved), the adjustment step is reduced to 0.3° / time, and the frame rate is reduced to 20 frames / second, which reduces the amount of shooting data and reduces the system operation pressure while ensuring the matching effect.

[0051] The trajectory data is collected by the camera module and matched with the operation trajectory to form a double verification mechanism of "trajectory + gesture"; for example, if a user wants to trigger a "flame special effect", he needs to complete both "drawing a circular trajectory" and "making a fist gesture", and only when the shape matching value of the two meets the standard can the trigger be successful, effectively reducing the situation of "similar gestures but accidental triggers" and "trajectory does not correspond but matching is successful"; dynamically adjusting the reference confidence value according to the shape matching value can flexibly adjust the judgment standard of subsequent gesture matching according to the actual recognition effect, such as increasing the judgment threshold when the trajectory and gesture deviation is large, and appropriately relaxing when the deviation is small, dynamically improving the overall matching precision and enhancing the stability in the interaction process. By adjusting the camera angle in real time through acceleration data, it can be ensured that no matter how the user moves or tilts the magic wand, the camera module can accurately aim at the posture sensor, avoiding trajectory capture omission or deviation; adjusting the angle step and frame rate according to the shape average value can not only improve the capture effect by increasing the frame rate and step size when the matching precision is insufficient, but also reduce the data volume when the precision meets the standard, achieving a balance between "effect and efficiency", making the trajectory interaction more smooth and the system operation more efficient.

[0052] In this embodiment, the experience area of the magic wand has four areas, ABCD. The magic wand has three shapes, including an initial shape, an intermediate shape, and a final shape. In the initial shape, the magic wand has no interaction point feedback, and is in a completely original state. In the intermediate shape, the magic wand has passed through 1-2 areas. In the final shape, the magic wand has completed all areas, and all the light beads are lit.

[0053] When the magic wand enters an area, it passes through the gateway at the entrance of the area, and triggering the touch button can reset the magic wand state to the initial shape. The magic wand records the reset date, and each magic wand can only be reset once a day. The magic wand is turned off by double-clicking the touch button, and the shutdown remembers the last shape unless it is reset through the entrance gateway. If the user's magic wand is already in the final shape, gesture recognition can still be performed. When gesture recognition is performed, the failure or success of the light effect can be played. When playing the gesture completion effect and the feedback light effect, whether other areas have been activated or not, when the magic wand is in the final shape, it remains lit unless a gesture is detected.

[0054] The complete operation process of the magic wand is as follows: 1. Turn on for 3 seconds, the boot effect is that the white 0th light bead slowly lights up in 1 second. Then the wand body of the magic wand flows and fills from the 1st light bead to the 18th light bead, the 18th light bead performs a 1-second blue gradient, and then all the lights are gradually extinguished in 1 second; the boot is accompanied by vibration until the lights are extinguished.

[0055] 2. When entering the corresponding area connected to the gateway, the white 1st light is always on when the corresponding area needs to press the touch area to emit a signal to the gateway.

[0056] 3, hold the touch area, white 0 lamp bead slowly breathing, from dark to light to dark, the cycle is 6 seconds.

[0057] 4, when collecting gestures 17-18 lamp beads light blue.

[0058] 5, the moment when the gesture is completed to achieve meteor shower lamp effect, 2-4 lamp beads are lit up white at the same time, and move quickly to 18, after 17-18 lamp beads stop, bright up and extinguish, while executing the lamp effect, short shock once.

[0059] Feedback after meteor shower lamp effect is completed: A: failed lamp effect, representing insufficient gesture samples, not matching the predetermined gesture, etc. Any unsuccessful situation, 17-18 lamp beads are quickly short flashed twice red, short shock once.

[0060] B: successful lamp effect, representing the gesture matching the gateway predetermined gesture, 17-18 lamp beads light blue, breathing lamp effect, bright and dark frequency from 5Hz to 100Hz, then from 18 lamp beads to 1 lamp beads direction, fill the corresponding lamp beads. Lightning gesture fills 2-6 lamp beads, triangular gesture fills 7-11 lamp beads, circular gesture fills 12-16 lamp beads, while executing the lamp effect, short shock twice.

[0061] 6, when multiple magic wands converge in the gateway in the trigger area D, the light gradually changes from white to light blue.

[0062] 7, the order of the user triggering the ABCD four trigger areas is indefinite, when completing the last trigger area, 1, 17 and 18 lamp beads gradually change to blue. Then, all the lights of the magic wand remain constant.

[0063] The embodiment also discloses an intelligent magic wand interaction system based on gyroscope gesture recognition, comprising a processor, wherein the processor executes the steps of the intelligent magic wand interaction method based on gyroscope gesture recognition as described in any one of the above.

[0064] Although the embodiments of the application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the application.

Claims

1. A smart wand interaction method based on gyroscopic gesture recognition, characterized in that, The method comprises the following steps: In response to the obtained starting instruction, real-time gesture posture is collected based on a preset posture sensor, and first posture data is generated in response to the obtained stopping instruction; the first posture data is converted into first gesture data using a preset first conversion algorithm, and a preset operation gesture is matched in a static database based on the first gesture data; If the operation gesture is matched, video content and light content corresponding to the operation gesture are called from a preset content database, the video content is played through a preset display device, and the light content is projected through a preset light device; Otherwise, a matching failure feedback is performed and the starting instruction is re-waited; In response to the matched operation gesture, real-time dynamic posture is collected based on the posture sensor, second posture data is generated, the second posture data is converted into second gesture data using a preset second conversion algorithm, and a preset action gesture is matched in a dynamic database based on the second gesture data; If the action gesture is matched, the action amplitude of the action gesture is calculated, and the playing speed of the video content or the projection range of the light content is adjusted according to the positive correlation of the action amplitude. 2.The smart magic wand interaction method based on gyroscopic gesture recognition according to claim 1, characterized in that, The first posture data comprises a plurality of first posture values, and the first conversion algorithm comprises the following steps: Based on the obtained abnormal threshold range, the difference between adjacent first posture values is calculated as a posture difference value, and if the posture difference value is within the abnormal threshold range, the average value of the two first posture values corresponding to the posture difference value is used to replace the two first posture values; Based on the collection cycle of the gesture posture, a filter window length corresponding to the collection cycle is obtained; The first posture data is window filtered based on the filter window length; According to the NNOM model, first gesture data with content as a graph is identified from the first posture data; or, the action direction is identified from the first posture data, the direction vector is extracted according to the action direction, the direction vector sequence is obtained by sorting the direction vector based on time sequence, and the first gesture data is matched from a preset graph database according to the direction vector sequence. 3.The smart magic wand interaction method based on gyroscopic gesture recognition according to claim 2, characterized in that, In the step of matching the preset operation gesture in the static database based on the first gesture data, the following sub-steps are further included: Gesture similarity values are calculated by sequentially calculating the similarity values between the first gesture data and the elements in the static database, and gesture confidence values are calculated according to the gesture similarity values and a preset reference similarity value; the maximum gesture confidence value is selected as the final gesture confidence value; If the final gesture confidence value is greater than a preset reference confidence value, the corresponding element is taken as the matched operation gesture and the operation gesture is outputted; A confidence ratio value is calculated by calculating the ratio of the final gesture confidence value and the reference confidence value, and the filter window length is adjusted according to the positive correlation of the confidence ratio value. 4.The smart magic wand interaction method based on gyroscopic gesture recognition of claim 3, wherein, The second posture data comprises a plurality of second posture values; and the second conversion algorithm comprises the following steps: The plurality of second posture values are divided into a plurality of posture groups in time sequence according to a preset data length; The average value of all second posture values in each posture group is calculated, the second posture dynamic value is obtained by subtracting the average value from all second posture values in each posture group, and the second posture data is updated using the second posture dynamic value; The one-way fluctuation interval of the second gesture data is calculated, and the one-way fluctuation interval is a range interval of a plurality of continuous data with the same motion trend; The interval quantity and interval interval of the one-way fluctuation interval are calculated, and the interval average value is calculated according to the interval interval; The quantity calculation value is calculated according to the interval quantity and the preset interval comparison quantity, and the interval calculation value is calculated according to the interval interval and the preset interval comparison interval; The second gesture data is set, the quantity calculation value and the interval calculation value are written into the second gesture data, and the second gesture data is output. 5.The smart magic wand interaction method based on gyroscopic gesture recognition according to claim 4, characterized in that, In the step of matching the preset motion gesture in the dynamic database based on the second gesture data, the following sub-steps are further included: The quantity calculation value and the interval calculation value are extracted from the second gesture data; If the quantity calculation value is less than the preset quantity setting value or the interval calculation value is greater than the preset interval setting value, the matching fails; Otherwise, a matching calculation value is calculated by weighted average according to the quantity calculation value and the interval calculation value; Gesture difference values are calculated by difference calculation between the matching calculation value and feature values corresponding to elements in the dynamic database one by one, the smallest gesture difference value is screened out, the corresponding element is taken as the matched motion gesture, and the motion gesture is output; The weighted value of the quantity calculation value is positively correlated with the gesture difference value, and the weighted value of the interval calculation value is negatively correlated with the gesture difference value. 6.The smart magic wand interaction method based on gyroscopic gesture recognition according to claim 5, characterized in that, The method further includes the following steps: A gesture volume library corresponding to the operation gesture and the motion gesture is obtained; An operation basic value is matched according to the operation gesture, and a motion basic value is matched according to the motion gesture; The execution duration of the operation gesture is calculated, and the duration of the motion gesture is calculated; Execution weight value and duration weight value are calculated by normalization according to the execution duration and the duration; An operation volume value is calculated according to the operation basic value and the execution weight value, a motion volume value is calculated according to the motion basic value and the duration weight value, a final volume value is calculated according to the operation volume value and the motion volume value, and the volume of the video content is controlled according to the final volume value. 7.The smart magic wand interaction method based on gyroscopic gesture recognition of claim 6, wherein, The method further includes the following steps: A matching ratio value is calculated according to the matching calculation value and the feature value corresponding to the element; The ratio of the confidence ratio value and the matching ratio value is an adjustment ratio value, the operation basic value is positively correlated with the adjustment ratio value, and the motion basic value is negatively correlated with the adjustment ratio value. 8.The smart magic wand interaction method based on gyroscopic gesture recognition of claim 7, wherein, The method further includes the following steps: A camera module of the alignment posture sensor is set, and the camera module collects the motion trajectory of the posture sensor to generate trajectory data based on a start instruction; An operation trajectory is matched from a preset trajectory database according to the trajectory data; If the operation trajectory is matched, a stop instruction corresponding to the start instruction is obtained, and a first operation gesture after the stop instruction is obtained; A shape matching value of the operation trajectory and the operation gesture is calculated, if the shape matching value is less than a preset setting matching value, it is prompted that the trajectory recognition does not correspond, a shape adjustment value is calculated according to the shape matching value and the setting matching value, and the reference confidence value is negatively correlated with the shape adjustment value. 9.The smart magic wand interaction method based on gyroscopic gesture recognition of claim 8, wherein, The method further includes the following steps: Acceleration data in the posture sensor is obtained in real time, and the shooting angle of the camera module is adjusted according to the acceleration data, so that the angle of the camera module corresponds to the posture of the posture sensor; The average value of the plurality of shape matching values is calculated to obtain a shape average value, and the angle adjustment step and the shooting frame rate of the camera module are adjusted according to the negative correlation of the shape average value.

10. A smart wand interaction system based on gyroscopic gesture recognition, characterized in that, The application relates to a processor, wherein the processor executes the steps of the smart magic wand interaction method based on the gyro gesture recognition as claimed in any one of claims 1-9.

Images