An intelligent magic wand interaction method and system based on gyroscope gesture recognition

Abstract

The application relates to the technical field of data interaction recognition, and discloses an intelligent magic wand interaction method and system based on a gyroscope gesture recognition, which comprises the following steps: in response to a starting instruction, gesture postures are collected, and in response to a stopping instruction, first posture data are generated; the first posture data are converted into first gesture data, and an operation gesture is matched in a static database based on the first gesture data; if the operation gesture is matched, corresponding video content and light content are called from a content database; otherwise, the starting instruction is re-waited; in response to the operation gesture, dynamic postures are collected, second posture data are generated, the second posture data are converted into second gesture data, and an action gesture is matched in a dynamic database based on the second gesture data; if the action gesture is matched, the action amplitude of the action gesture is calculated, and the playing speed of the video content or the projection range of the light content is adjusted according to the positive correlation of the action amplitude, so that the interaction delay time can be shortened, and the interaction effect can be improved.

Description

Technical Field

[0001] This application relates to the technical field of data interaction recognition, and in particular to a smart wand interaction method and system based on gyroscope gesture recognition. Background Technology

[0002] With the rapid development of digital entertainment, education and training, virtual simulation, and other fields, users' demand for immersive content experiences is becoming increasingly urgent. Traditional human-computer interaction methods, such as keyboards, mice, and touchscreens, are unable to simulate natural interaction behaviors in real-world scenarios and cannot meet users' requirements for real-time interaction, accuracy, and immersion in immersive environments. Therefore, developing an immersive content experience interaction system that can achieve natural and efficient interaction has become crucial for improving user experience and expanding the application scenarios of immersive technologies, and has significant practical implications and market value.

[0003] Currently, mainstream immersive content experience interactive systems mainly consist of an image acquisition module and a display output module. The image acquisition module is centered around a camera, while the display output module is centered around a monitor. The camera, as the core sensing component, captures visual information such as the user's body posture and movement trajectory in real time, converts it into digital signals, and transmits them to the system processing unit. The processing unit analyzes and recognizes the acquired image data, generates corresponding control commands, and drives the monitor to play dynamic visuals and synchronized sound that match the user's movements. This is commonly found in VR / AR device systems, interactive projection entertainment devices, and other application scenarios.

[0004] However, immersive content experience interactive systems based on cameras require multiple complex processing steps such as target detection, feature extraction, and action matching for continuous frame images captured by the camera. The large amount of image data and the long processing time result in significant delays in the generation of interactive commands. It is difficult to synchronize user actions with the display feedback, which seriously affects the immersive experience. When multiple people or targets are present, problems such as target occlusion and feature confusion are likely to occur in the images captured by the camera. The system has difficulty accurately identifying and tracking the actions of the designated interactive user, resulting in a significant drop in recognition accuracy and even interaction failure, which further increases the interaction delay time. Summary of the Invention

[0005] To shorten the latency of interactive systems and improve the interactive effect, this application provides a smart wand interaction method and system based on gyroscope gesture recognition.

[0006] Firstly, this application provides a smart wand interaction method based on gyroscope gesture recognition, employing the following technical solution:

[0007] A smart wand interaction method based on gyroscope gesture recognition includes the following steps:

[0008] In response to the acquired start command, the gesture posture is collected in real time based on the preset posture sensor; in response to the acquired stop command, the first posture data is generated; the first posture data is converted into first gesture data using the preset first conversion algorithm; and the preset operation gesture is matched in the static database based on the first gesture data.

[0009] If a gesture is matched, the system retrieves the corresponding video and lighting content from the preset content database, plays the video content through the preset display device, and projects the lighting content through the preset lighting device; otherwise, it sends a matching failure feedback and waits for a restart command.

[0010] In response to the matched operation gesture, dynamic posture is collected in real time based on the posture sensor to generate second posture data. The second posture data is converted into second gesture data using a preset second conversion algorithm. Based on the second gesture data, a preset action gesture is matched in the dynamic database.

[0011] If a gesture is matched, the amplitude of the gesture is calculated, and the playback speed of the video content or the projection range of the lighting content is adjusted according to the positive correlation of the amplitude.

[0012] By adopting the above technical solution, in response to the acquired start command, the system collects hand gestures in real time based on a preset posture sensor, and generates first posture data in response to the acquired stop command. Subsequently, the first posture data is converted into first gesture data using a preset first conversion algorithm. Then, based on the first gesture data, a preset operation gesture is matched against a static database. This process eliminates the need for complex processing steps such as target detection and feature extraction on continuous frame images captured by the camera, avoiding the problems of large image data volume and long processing time. It shortens the time required to generate interactive commands, keeps video and lighting content synchronized with user actions, and improves the immersive experience. When an operation gesture is matched, it is retrieved from a preset content database. The system invokes video and lighting content corresponding to the operation gestures, plays the video content through a preset display device, and projects the lighting content through a preset lighting device, thus realizing the basic presentation of immersive content interaction. At the same time, in response to the matched operation gestures, it generates second posture data by collecting dynamic postures in real time based on posture sensors. The second posture data is converted into second gesture data using a preset second conversion algorithm. Based on the second gesture data, preset action gestures are matched in a dynamic database. By calculating the amplitude of the action gestures, the playback speed of the video content or the projection range of the lighting content is adjusted according to the positive correlation of the amplitude of the action gestures, making the correlation between operation and feedback during the interaction stronger and improving the coherence and adaptability of content interaction.

[0013] Furthermore, the first attitude data includes multiple first attitude values, and the first conversion algorithm includes the following steps:

[0014] Based on the obtained abnormal threshold range, the difference between adjacent first attitude values ​​is calculated as the attitude difference. If the attitude difference is within the abnormal threshold range, the average value of the two first attitude values ​​corresponding to the attitude difference is used to replace the two first attitude values.

[0015] Based on the acquisition period of the acquired gestures, obtain the length of the filter window corresponding to the acquisition period;

[0016] Window filtering is performed on the first attitude data based on the filter window length;

[0017] The first gesture data, which is a graphic, is identified from the first posture data according to the NNOM model; or, the direction of the action is identified from the first posture data, the direction vector is extracted according to the direction of the action, the direction vector is sorted according to the time sequence to obtain the direction vector sequence, and the first gesture data is matched from the preset graphic database according to the direction vector sequence.

[0018] By adopting the above technical solutions, anomaly processing and window filtering are performed on the first posture data to maintain data stability. This facilitates the matching of the first gesture data with the NNOM model or direction vector sequence, making the gesture matching process smoother, maintaining the timeliness of interactive feedback, enhancing the continuity of immersive content experience, and making the association between gestures and feedback more in line with expectations.

[0019] Furthermore, the step of matching preset operation gestures in the static database based on the first gesture data also includes the following sub-steps:

[0020] The gesture similarity value is calculated by comparing each gesture with an element in the static database. The gesture confidence value is then calculated based on the gesture similarity value and a preset reference similarity value. The gesture confidence value with the highest value is selected as the final gesture confidence value.

[0021] If the final gesture confidence value is greater than the preset reference confidence value, the corresponding element will be used as the matched operation gesture and the operation gesture will be output.

[0022] The confidence ratio is calculated as the ratio of the final gesture confidence value to the reference confidence value. The length of the filter window is then adjusted based on the positive correlation of the confidence ratio.

[0023] By adopting the above technical solution, calculating gesture similarity values ​​and gesture confidence values, and filtering the gesture confidence value with the highest confidence value, the accuracy of gesture matching can be improved and mismatches can be reduced. Adjusting the length of the filtering window based on the confidence ratio can maintain the adaptability of data processing and gesture recognition, making the matching of the first gesture data and the gesture more stable, providing a more appropriate gesture basis for subsequent calls to video and lighting content, and improving the immersive interactive experience.

[0024] Furthermore, the second attitude data includes multiple second attitude values; the second conversion algorithm includes the following steps:

[0025] According to the preset data length, multiple second attitude values ​​are divided into multiple attitude groups in chronological order;

[0026] Calculate the average of all second attitude values ​​in each attitude group, subtract the average from all second attitude values ​​in each attitude group to obtain the second attitude dynamic value, and use the second attitude dynamic value to update the second attitude data;

[0027] Calculate the unidirectional fluctuation range of the second attitude data. The unidirectional fluctuation range is the range of multiple consecutive data with the same movement trend.

[0028] Calculate the number of unidirectional fluctuation intervals and the interval intervals, and calculate the average interval based on the interval intervals, where the interval interval is the time interval between unidirectional fluctuation intervals;

[0029] The quantity calculation value is calculated based on the number of intervals and the preset number of interval comparisons, and the interval calculation value is calculated based on the interval interval and the preset interval comparison interval.

[0030] Set the second gesture data, write the quantity calculation value and the interval calculation value into the second gesture data, and output the second gesture data.

[0031] By adopting the above technical solution, the quantity calculation value and interval calculation value are obtained by processing the second posture data and written into the second gesture data. This helps to make the second gesture data fit the actual dynamic posture, maintain the integrity of the second gesture data, provide an adaptation basis for matching action gestures in the dynamic database, reduce data deviation interference, and improve the smoothness of interactive feedback.

[0032] Furthermore, the step of matching preset gestures in the dynamic database based on the second gesture data also includes the following sub-steps:

[0033] Extract the quantity and interval calculation values ​​from the second gesture data;

[0034] If the calculated quantity is less than the preset quantity setting or the calculated interval is greater than the preset interval setting, the matching fails; otherwise, the matching calculated value is calculated by weighted average of the calculated quantity and the calculated interval.

[0035] The gesture difference is obtained by calculating the difference between each matching value and the feature value corresponding to each element in the dynamic database; the element with the smallest gesture difference is selected and the corresponding element is used as the matched action gesture and the action gesture is output.

[0036] The weighted values ​​are calculated based on the positive correlation of gesture differences and the weighted values ​​based on the intervals of the negative correlation.

[0037] By adopting the above technical solution, extracting numerical judgments and calculating matching values, and filtering the smallest gesture difference, it is beneficial to improve the accuracy of action gesture matching and reduce mismatches; adjusting the weighting value based on the gesture difference can maintain the adaptability of subsequent matching and make action gesture matching more stable.

[0038] Furthermore, the method also includes the following steps:

[0039] Obtain the gesture volume library corresponding to operation gestures and action gestures;

[0040] The basic operation value is matched based on the operation gesture, and the basic action value is matched based on the action gesture;

[0041] Calculate the execution duration of the gesture and the duration of the action gesture;

[0042] The execution weight and duration weight are calculated by normalizing the execution duration and duration.

[0043] The operation volume value is calculated based on the operation base value and execution weight value. The action volume value is calculated based on the action base value and duration weight value. The final volume value is calculated based on the operation volume value and action volume value. The volume of the video content is controlled based on the final volume value.

[0044] By adopting the above technical solution, the final volume value is obtained by matching the basic values ​​of operation and action based on the gesture volume library and combining the duration calculation weight. This makes the volume control more in line with the gesture operation. At the same time, it maintains the correlation between volume adjustment and gesture execution and duration, improves the adaptability of volume control in interaction, and provides dynamically adapted volume output for video content playback.

[0045] Furthermore, the method also includes the following steps:

[0046] The matching ratio is calculated based on the matching calculated value and the feature value of the corresponding element;

[0047] The adjustment ratio is calculated as the ratio of the confidence ratio to the matching ratio. The adjustment base value is adjusted based on the positive correlation of the adjustment ratio and the adjustment base value is adjusted based on the negative correlation of the adjustment ratio.

[0048] By adopting the above technical solution, the base value is adjusted by adjusting the ratio, which helps to improve the compatibility between the base value and the gesture, maintain the dynamics of volume calculation, make the final volume control more adaptable to the interaction needs, and enhance the matching degree between gesture and volume adjustment.

[0049] Furthermore, the method also includes the following steps:

[0050] A camera module is set up to align with the attitude sensor. Based on the start command, the camera module collects the motion trajectory of the attitude sensor to generate trajectory data.

[0051] The operation trajectory is matched with the pre-set trajectory database based on the trajectory data;

[0052] If an operation trajectory is matched, the stop command corresponding to the start command is obtained, and the first operation gesture after the stop command is obtained;

[0053] Calculate the shape matching value between the operation trajectory and the operation gesture. If the shape matching value is less than the preset matching value, prompt that the trajectory recognition does not correspond. Calculate the shape adjustment value based on the shape matching value and the preset matching value, and adjust the reference confidence value based on the negative correlation of the shape adjustment value.

[0054] By adopting the above technical solution, the camera module collects trajectory data and matches the operation trajectory, which facilitates the correspondence between the operation trajectory and the operation gesture, and reduces the situation of trajectory recognition mismatch. Adjusting the reference confidence value according to the shape matching value can maintain the adaptability of subsequent gesture matching, dynamically improve the overall matching accuracy, and enhance the matching stability during the interaction process.

[0055] Furthermore, the method also includes the following steps:

[0056] The camera module's shooting angle is adjusted based on the acceleration data from the attitude sensor in real time, so that the angle of the camera module corresponds to the attitude of the attitude sensor.

[0057] The average of the most recent multiple shape matching values ​​is calculated to obtain the shape average value. The angle of the camera module is adjusted, the step size and the shooting frame rate are adjusted according to the negative correlation of the shape average value.

[0058] By adopting the above technical solutions, the camera angle is adjusted by using acceleration data, which helps to maintain its correspondence with the attitude sensor angle; the step size and frame rate are adjusted by the shape average value, which facilitates dynamic adaptation, improves trajectory matching effect, and reduces the amount of shooting data.

[0059] Secondly, this application provides a smart wand interaction system based on gyroscope gesture recognition, which adopts the following technical solution:

[0060] A smart wand interaction system based on gyroscope gesture recognition includes a processor, wherein the processor executes the steps of the smart wand interaction method based on gyroscope gesture recognition as described in any one of the preceding claims. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the method of smart wand interaction based on gyroscope gesture recognition.

[0062] Figure 2 This is a schematic diagram of the logical flow of the NNOM model algorithm.

[0063] Figure 3 This is a schematic diagram of the projection processing flow.

[0064] Figure 4 This is a business logic flow diagram. Detailed Implementation

[0065] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0066] This application discloses a smart wand interaction method based on gyroscope gesture recognition. The hardware device of the smart wand includes an external structural component and internal functional modules. The external structural component includes a top component, a wand body component, and a bottom component. The top component has a fluorescent column, which integrates a 940nm infrared LED light as an infrared recognition light source carrier, forming an infrared camera component. The wand body component has a white light-transmitting area, a touch area, a double-sided PCB, a main control board mounting position, and a rechargeable battery arranged sequentially along the axial direction. The surface of the white light-transmitting area is coated with PT:4271C to ensure light transmission performance. A constantly lit LED bead is set at the corresponding position of the touch area as a touch interaction indicator. The double-sided PCB is equipped with 13 RGB LED single-sided panels for wand body lighting output. The main control board mounting position is used to install the main control chip. The rechargeable battery is a 14500 model, providing hardware power. The wand body shell is coated with PT:7596C, with a matte black base color, and some areas are treated with two anti-matte paint treatments and anti-fouling processes; one RGB LED bead is located at the bottom of the wand body. The bottom component uses resin crystal parts as structural finishing and exterior decoration.

[0067] The internal functional modules include an MCU hardware control core, a Bluetooth 4.2 transceiver module, a 6-axis IMU module, RGB LED patch lights, a 940nm infrared LED, a vibration module, and a touch module. Each module is electrically connected to the main control board and is uniformly scheduled by the MCU. The hardware workflow is implemented by the MCU in coordination with each module. Power-on is triggered by vibration. After power-on, the RGB LEDs on the cane execute a flowing light effect, and the RGB LED at the top executes a breathing light effect, generating initial feedback. Three seconds after power-on, the Bluetooth 4.2 module enters a constant connection state to establish a link with the paired smart mirror device. Simultaneously, the 940nm infrared LED remains constantly lit, providing an infrared recognition light source for the paired device, enabling communication and recognition preparation. During the hardware's execution of gesture recognition, when the cane is detected to be waving, the 6-axis IMU module starts data acquisition, recognizing gestures such as flashing, flicking, spinning, and heart shapes. The hardware self-testing process is as follows: First, the touch area press state is detected. If the action pauses for more than 100ms, the speed threshold is calculated, and IMU acquisition is initiated. The waving duration must be ≤30ms and the effective sample size must be met before proceeding to the next step; otherwise, it is considered an invalid gesture, and no feedback is executed. In the idle state, the top RGB LED displays a breathing light effect, and the brightness of the light effect adaptively adjusts according to the signal strength. During gesture acquisition, the top LED is blue. After valid acquisition, the cane body LED plays a flowing light effect, and the top LED turns off after it ends. When valid gesture inference is completed, the vibration module performs vibration to achieve sound, light, and tactile feedback. Valid gesture data is sent to the Magic Mirror device via Bluetooth. The Magic Mirror device is a paired display device. After pairing, the Magic Mirror device sends back the results and executes the corresponding lighting effects based on the results. When the battery level is lower than the threshold, the MCU controls automatic shutdown to achieve interaction and low battery protection.

[0068] Reference Figure 1 The method includes the following steps: In response to the acquired start command, gesture posture is acquired in real time based on a preset posture sensor. For example, gesture posture data acquisition uses a microcontroller and posture sensor mounted on the smart wand. The microcontroller can be an STM32, and the posture sensor can be a magnetic field sensor and an MPU6050 or BMI160. The posture sensor is connected to the MCU and acquires data at a fixed frequency. Accelerometer data includes acc_x, acc_y, and acc_z. Gyroscope data includes gyro_x, gyro_y, and gyro_z. Magnetic field sensor data includes mag_x, mag_y, and mag_z. Alternatively, the posture data fused from the accelerometer and gyroscope can be directly acquired through the DMP motion calculation unit built into the posture sensor, and then further fused with the magnetic field sensor data can be selected. Alternatively, an infrared camera assembly can be formed using infrared LEDs and an infrared camera to recognize gesture postures via infrared.

[0069] In response to the received stop command, first posture data is generated, which includes multiple first posture values. The first posture data is converted into first gesture data using a preset first conversion algorithm. This algorithm can obtain an anomaly threshold range and calculate the difference between adjacent first posture values ​​as the posture difference. If the posture difference is within the anomaly threshold range, the average of the two first posture values ​​corresponding to the posture difference is used to replace those two values, thus enabling the identification and replacement of anomalies in the first posture data. Based on the acquisition period of the gesture posture, the length of the filtering window corresponding to the acquisition period is obtained. Window filtering is performed on the first posture data based on this filtering window length, which can smooth the fluctuations in the first posture data.

[0070] Assuming the smart wand captures a "standstill" stop gesture, and the attitude sensor (such as MPU6050) collects acceleration data at a period of 10ms / time, the data generated after removing the g-force effect after triggering the stop command is: [1.1, 1.2, 1.3, 3.2, 1.4, 1.3], where 3.2 is an outlier caused by momentary hand tremors. Using the first conversion algorithm, the outlier threshold range of ±0.5m / s is first obtained. 2 Calculate the difference between adjacent poses: 1.2-1.1=0.1 (within the threshold, remains unchanged), 1.3-1.2=0.1 (remains unchanged), 3.2-1.3=1.9 (exceeds the threshold). Then replace these two values ​​with the average of 1.3 and 3.2, 2.25, and the data becomes [1.1,1.2,2.25,2.25,1.4,1.3]. Next, calculate the next difference: 2.25-1.4=0.85 (still exceeds the threshold), and replace it with the average of 2.25 and 1.4, 1.825. The final corrected data is [1.1,1.2,2.25,1.825,1.825,1.3]. Then, based on the 10ms acquisition period, the filter window length was determined to be 4. The corrected data was then subjected to a moving average filter: the average value of the first 4 data points (1.1+1.2+2.25+1.825) / 4≈1.59. Subsequent windows were calculated sequentially, and the smoothed first gesture data [1.1,1.2,1.59,1.69,1.64,1.3] was finally obtained, which eliminated outliers and small fluctuations and better matched the "stillness" gesture characteristics.

[0071] In this embodiment, a first gesture data containing a graphic is identified from the first pose data based on the NNOM model. For example... Figure 2 The algorithm logic around the NNOM model is as follows: first, sensor data is collected, then data preprocessing is performed, feature extraction is performed, the NNOM model is used for inference, gestures are classified, and then confidence is detected. If the confidence exceeds the threshold, the special effect is triggered; otherwise, the gesture is ignored.

[0072] After data cleaning, data alignment and segmentation are performed first, dividing each gesture sample into a fixed-length time window, such as 50 time steps, to ensure all samples are of consistent length. Then, feature extraction is performed, directly training the model using the raw time-series data. For example, LSTM is used to calculate statistical features, converting the time-series data into a fixed-dimensional feature vector: mean, standard deviation, extreme values, peak values, correlation, etc. For example, for a 50-step × 9-axis gesture sample, we can calculate 5 statistical features for each axis, ultimately obtaining a 45-dimensional feature vector. Model training is then implemented using Keras. Since the input is a feature vector or a time-series sequence, different model architectures can be chosen. Model Architecture A: Using a Multilayer Perceptron (MLP) – suitable for feature vector-based inputs; Model Architecture B: Using a One-Dimensional Convolutional Network (1D-CNN) – suitable for raw time-series data. Similar to the image classification process, the trained Keras model is transformed using an NNOM model for real-time inference. Not all features are beneficial for classification; feature importance analysis, such as random forests, can be used to select the most critical features, reducing input dimensionality and computational cost. The calibration data used for NNOM model conversion can represent the true input distribution. Feature extraction algorithms implemented on MCUs often use integer operations instead of floating-point operations.

[0073] In another embodiment, the action direction is identified from the first posture data, a direction vector is extracted based on the action direction, and the direction vectors are sorted according to time sequence to obtain a direction vector sequence. The first gesture data is then matched from a preset graphics database based on the direction vector sequence. Anomaly handling and window filtering are performed on the first posture data to maintain data stability, which facilitates the matching of the first gesture data by the NNOM model or the direction vector sequence, making the gesture matching process smoother, maintaining the timeliness of interactive feedback, enhancing the coherence of the immersive content experience, and making the association between gestures and feedback more in line with expectations.

[0074] The following calculations can be performed on the gateway, backend, or host computer, which runs on the field PC. In this embodiment, the gateway is used as the core for gesture recognition, and the host computer is used as the core for controlling the field equipment. Based on the first gesture data, preset operation gestures are matched against a static database. Operation gestures in the static database can be defined as: waving a stick (swinging left and right), drawing a shape (changing the direction of a broken line), spinning (rotating around an axis), etc. A gesture similarity value is calculated by comparing each element in the static database with the first gesture data. A gesture confidence value is then calculated based on the gesture similarity value and a preset reference similarity value. The largest gesture confidence value is selected as the final gesture confidence value. If the final gesture confidence value is greater than the preset reference confidence value, the corresponding element is used as the matched operation gesture and the gesture is output. The ratio of the final gesture confidence value to the reference confidence value is calculated as the confidence ratio, and the filter window length is adjusted based on the positive correlation of the confidence ratio. By calculating gesture similarity values ​​and gesture confidence values, and filtering for the highest gesture confidence value, the accuracy of gesture matching is improved, reducing mismatches. Adjusting the filter window length based on the confidence ratio maintains the adaptability of data processing and gesture recognition, making the matching of the first gesture data and the operation gesture more stable. This provides a more suitable gesture basis for subsequent calls to video and lighting content, enhancing the immersive interactive experience. The gateway is a key node connecting the wand, the host computer, and the matching display device. It is the core control channel of the smart wand and can switch between multiple working modes according to different interaction needs. If the working mode is gesture gateway, it first notifies the wand that it is currently in the gateway type, then verifies the received wand gesture data. It also supports adjusting the wand's Bluetooth transmission power via USB serial port, with 10 adjustable levels. During operation, it compares the wand's gesture data with a static database. If a match is found, it sends a confirmation command and the corresponding operation gesture to the wand; if a mismatch occurs, there is no response. Simultaneously, it reports the gesture verification result and the corresponding wand's MAC address to the host computer, ensuring the accuracy of gesture interaction. Specifically, when the wand enters the gateway's range, it receives a gateway-type signal. The gateway only listens for the corresponding gesture from the wand; other gestures are invalid. When the gateway receives a successful and matched wand gesture, it reports to the PC and sends a wand feedback command. In response to the feedback command, the wand controls the LED beads that have a mapping relationship with the corresponding gesture to stay lit.

[0075] Specifically, once the wand gesture is completed, a request is sent to the gateway, containing the wand's MAC address. The gateway responds with the received MAC address, also containing the wand's MAC address. The wand checks if the MAC address matches its own. If they match, the wand sends the gesture result to the gateway. The request corresponding to the gesture result includes the wand's MAC address and the gesture result. Upon receiving the request, the gateway determines if the gesture matches the predetermined one and responds to the wand. Regardless of whether the gesture matches, the request is reported to the host computer, which then executes the request and determines whether to activate the field device.

[0076] If there are multiple smart wands, the operating mode is a pairing gateway. First, the gateway notifies the wand that it is currently in gateway type, synchronizes the current pairing mode status with the wand, and identifies a 5-second baseline time window (which can be extended to 10 seconds). Within this window, it checks if multiple wands have established a communication connection. When the conditions for successful interaction among multiple wands within the set time are met, the pairing configuration process automatically starts. The communication time limit can also be set via USB serial port. Finally, the gateway reports the MAC addresses of all wands participating in the pairing to the host computer, supporting collaborative interaction among multiple wands. Specifically, when a wand enters the gateway's range, it receives a gateway type signal. Within the baseline time window, the gateway identifies whether three wands are simultaneously in an interactive state. If successful, it reports to the PC and the wands, and the top RGB LED of the wand lights up continuously.

[0077] If a reset is required, the working mode is to reset the gateway. First, the gateway notifies the wand that it is currently in the gateway type, receives instructions from the host computer, and sends a reset signal to the wand. After receiving the execution instruction, it issues a reset instruction. Upon receiving the instruction, the wand will switch the corresponding lighting effect feedback according to its own state. At the same time, the gateway will report the MAC address of the target wand to achieve a quick reset of the wand's state.

[0078] If a gesture is matched, the system retrieves the corresponding video and lighting content from the preset content database, plays the video content through a preset display device, and projects the lighting content through a preset lighting device. Otherwise, it provides a matching failure feedback and waits for a restart command. The display device uses a projector and a screen, and the lighting device uses a laser projector and the wand's built-in lighting effects. The matching failure feedback is provided by a built-in vibration motor. The microcontroller is connected to a wireless module to wirelessly connect to the backend. It is used to retrieve video content from the backend and display it on the screen via the projector. It also retrieves linear lighting effects and projects them onto the ground or wall via laser projection, and controls the LED beads or LED strips on the wand to display lighting effects. Assuming the user performs a "draw a triangle" gesture while holding the smart wand, after the system successfully matches the gesture in the static database, the microcontroller establishes a connection with the backend via the wireless module and retrieves the preset "Magic Triangle Blooming" themed video and blue-purple laser lighting effects from the content database. The system then instructs the projector to project the video onto the screen, displaying a rotating triangular magic circle. Simultaneously, a laser projection creates a shimmering linear triangular light effect on the ground, and the wand's built-in RGB LEDs light up with a flowing blue-purple light, creating an immersive interactive effect. If the user makes a random shaking gesture without a pre-defined correspondence, the system will fail to match the gesture. The wand's built-in vibration motor will then provide brief tactile feedback, indicating that the gesture was not recognized. The system will then re-enter a waiting state, awaiting the user to trigger the start command again and perform a valid gesture.

[0079] like Figure 3As shown, the projector's processing flow revolves around a closed-loop design of "standby-trigger-feedback-reset," maintaining consistently high efficiency. In normal operation, the projector is in standby mode, looping the "Fantasy Magic Circle Standby" video. A light blue light effect slowly rotates in the image, creating an immersive atmosphere, while the background system wirelessly monitors the smart wand's gestures in real time. If the user makes random movements without a pre-defined correspondence, or if the gesture is blurry and recognition fails, the projector will continue playing the standby video without interrupting the monitoring function. Once the user completes a valid "draw a circle" gesture and it matches successfully, the projector immediately stops the standby video and seamlessly switches to the "Crystal Ball Blooming" special effects video. The circular magic circle explodes, scattering starlight. After the 15-second special effects video finishes playing, the projector automatically returns to standby mode, awaiting the next user trigger.

[0080] In response to the successfully matched "draw a circle" gesture, the wand's built-in BMI160 attitude sensor immediately activates dynamic acquisition mode, capturing the user's subsequent dynamic postures at a high-frequency acquisition cycle of 10ms / time. For example, after the user draws a circle, they quickly wave the wand left and right. The sensor continuously acquires 12 sets of fused acceleration and angular velocity data, generating second attitude data containing 12 second attitude values: [1.8, 2.1, 2.4, 2.2, 1.9, 1.6, 1.3, 1.5, 1.7, 2.0, 2.3, 2.1]. According to the preset data length of 6 (i.e., every 6 consecutive values ​​form a group), the system divides the second attitude values ​​into two attitude groups in chronological order: the first group [1.8, 2.1, 2.4, 2.2, 1.9, 1.6] and the second group [1.3, 1.5, 1.7, 2.0, 2.3, 2.1]. Then, the average value of all second attitude values ​​in each group is calculated. The average value of the first group is (1.8+2.1+2.4+2.2+1.9+1.6) / 6=2.0, and the average value of the second group is (1.3+1.5+1.7+2.0+2.3+2.1) / 6=1.8. Subtract the corresponding average value from each second attitude value in each group to obtain the dynamic values ​​of the second attitude (first group: [-0.2,0.1,0.4,0.2,-0.1,-0.4]; second group: [-0.5,-0.3,-0.1,0.2,0.5,0.3]). The original second attitude data is updated with these dynamic values ​​to highlight the core features of attitude change.

[0081] Based on the updated second posture data, the system further analyzes the movement trend and identifies unidirectional fluctuation ranges; that is, data ranges with continuous movement trends (upward or downward). After calculation, three unidirectional fluctuation ranges were finally determined: [-0.2, 0.1, 0.4, 0.2] (upward trend), [-0.1, -0.4, -0.5, -0.3, -0.1] (downward trend), and [0.2, 0.5, 0.3] (upward trend). The total number of ranges was 3, and the intervals (time intervals between two adjacent unidirectional fluctuation ranges) were 20ms (corresponding to 2 acquisition cycles) and 20ms respectively. The average interval was calculated to be (20+20) / 2=20ms. Next, based on the preset interval comparison quantity of 4 and interval comparison interval of 30ms, the quantity calculation value is obtained by normalization calculation = interval quantity / interval comparison quantity = 3 / 4 = 0.75, and the interval calculation value is obtained by interval average / interval comparison interval = 20 / 30 ≈ 0.67. These two core parameters are written into the second gesture data and output to provide accurate basis for subsequent action matching.

[0082] The system extracts a quantity calculation value of 0.75 and an interval calculation value of 0.67 from the output second gesture data. Based on preset rules, a preliminary judgment is made: the preset quantity setting is 0.5 (below this value indicates an unclear movement trend), and the preset interval setting is 1.0 (above this value indicates poor movement continuity). Since 0.75 ≥ 0.5 and 0.67 ≤ 1.0, the matching condition is satisfied. Subsequently, according to the preset weighting ratio (quantity calculation value weight 0.6, interval calculation value weight 0.4), the matching calculation value is calculated as: 0.75 × 0.6 + 0.67 × 0.4 ≈ 0.718. The calculated matching value is compared one by one with the preset gesture feature values ​​in the dynamic database. The dynamic database includes gestures such as "rapid left and right waving" (feature value 0.7), "slow up and down swaying" (feature value 0.3), and "uniformly drawing circles" (feature value 0.5). The calculated gesture differences are 0.018, 0.418, and 0.218, respectively. The gesture with the smallest difference of 0.018, "rapid left and right waving," is selected as the successfully matched gesture. Simultaneously, according to the rule of "adjusting the weighted value of the quantity calculation for positive correlation of gesture difference and adjusting the weighted value of the interval calculation for negative correlation," the weighted value of the quantity calculation is finely adjusted from 0.6 to 0.62, and the weighted value of the interval calculation is finely adjusted from 0.4 to 0.38, optimizing the parameter fit for subsequent matching.

[0083] After matching the "rapid left and right waving" gesture, the system calculates the amplitude of the movement as 3.5 based on the peak difference of the second posture dynamic value (the value ranges from 0 to 5, with larger values ​​indicating more intense movements). According to the rule that "movement amplitude is positively correlated with adjustment effect," the playback speed of the "crystal ball blooming" special effects video is increased from 1x to 1.9x, making the rhythm of the image correspond to the intensity of the movement; at the same time, the projection range of the laser projection is expanded from the initial 2.5-meter radius to 4.5 meters, covering a larger interactive space. Simultaneously, the system calls the preset gesture volume library, matching the base value of the "draw a circle" gesture to 65 (volume scale 0-100), and the base value of the "quick left and right waving" gesture to 75. The execution duration of the gestures is calculated to be 2.2 seconds, and the duration of the gestures to be 1.8 seconds. After normalization (execution weight = 2.2 / (2.2+1.8) = 0.55, duration weight = 1.8 / (2.2+1.8) = 0.45), the weight parameters are obtained. Combining the previously calculated matching ratio of 0.718 to the corresponding feature value of 0.7 (≈1.026), and the confidence ratio of 1.15 for gesture matching, the adjustment ratio is calculated as 1.15 / 1.026 ≈ 1.121. The base value is adjusted according to the rules: positive correlation (65 × 1.121 ≈ 72.87), negative correlation (75 ÷ 1.121 ≈ 66.91). The final calculated operation volume value is 72.87 × 0.55 ≈ 40.08, and the action volume value is 66.91 × 0.45 ≈ 30.11. The sum of these values ​​gives a final volume value of ≈ 70.19. The system automatically adjusts the video volume to 70, achieving deep adaptation between volume and gesture operation and dynamic action.

[0084] The core advantage of the entire interaction process lies in "redundant data processing": upon responding to the start command, the posture sensor directly collects the gesture posture, generates the first posture data, and converts it into the first gesture data, eliminating the need for complex operations such as target detection and feature extraction of continuous frame images as required by traditional camera solutions. This design avoids the problems of large image data volume and long algorithm time, reducing the latency of interaction command generation from over 500ms in traditional solutions to within 100ms, ensuring real-time synchronization between video playback, light projection, volume adjustment, and user actions, significantly improving the immersive experience. When an operation gesture is matched, the system quickly calls up the corresponding video and lighting content to complete the basic presentation of immersive interaction; subsequently, through dynamic posture acquisition, data conversion, action matching, and multi-dimensional adjustment, the correlation between operation and feedback is strengthened; for example, the greater the amplitude of the action, the faster the video, the wider the lighting range, and the higher the volume, forming a closed-loop adaptation of "action-feedback," significantly improving the coherence and adaptability of content interaction.

[0085] The overall business logic is as follows Figure 4As shown, visitors can draw gestures using a handheld magic wand device. The microcontroller on the magic wand device collects sensor data from the posture sensor and sends the data to the main system in the background. The main system performs gesture recognition processing. If the recognition is successful, it invokes a media system containing a projector and laser projection to trigger special effects videos and lighting effects, displaying multimedia effects to the visitor. If the recognition fails, it sends vibration feedback and plays a failure message through the media system.

[0086] To provide negative feedback on the gesture recognition results and further improve interaction accuracy, a dedicated camera module can be set up to align with the wand's posture sensor (such as the BMI160). If the application scenario is dimly lit (such as in an immersive escape room), a 940nm infrared camera component can be used to avoid ambient light interference. In brightly lit interactive exhibition halls, a high-definition visible light camera component is used to ensure clear trajectory capture. With the addition of the camera module, the system gateway's operating mode also adds a "camera gateway": when a user enters an interactive area requiring trajectory verification, the camera gateway first sends a "currently in camera gateway mode" command to the wand via Bluetooth. Simultaneously, it automatically connects to the external camera module and the backend trajectory recognition program. It does not participate in complex data calculations; it only forwards the wand's unique MAC address (such as "AA:BB:CC:DD:EE:FF") to the backend, providing stable data transmission support for interactive scenarios requiring both "trajectory + gesture" verification. The entire gateway system can flexibly switch between gesture gateway, pairing gateway, reset gateway and camera gateway: switch to gesture gateway to ensure accurate recognition when operating alone, switch to pairing gateway to realize multi-wank linkage when multiple people are interacting collaboratively, use reset gateway to quickly restore the state when the device is abnormal, and switch to camera gateway to expand the interaction dimension when trajectory verification is required. This allows the smart wand system to adapt to both single-person immersive experience and meet the needs of different scenarios such as multi-person collaboration and trajectory interaction, greatly improving the overall compatibility and flexibility.

[0087] Based on the user-triggered start command (such as pressing and holding the wand touch area for 2 seconds), the camera module immediately starts and aligns with the posture sensor to capture the user's wand movement trajectory in real time. For example, if the user draws a "pentagram" in the air, the camera module will continuously capture key points of the trajectory, generating trajectory data containing coordinates and timing information. The system matches this trajectory data with a preset trajectory database (containing standard trajectory templates such as "pentagram," "triangle," and "circle"). If a "pentagram" operation trajectory is successfully matched, the system automatically obtains the stop command corresponding to this start command (the user releases the touch area) and extracts the first operation gesture after the stop command (such as the user holding the wand still after drawing the pentagram). The system then calculates the shape matching value between the operation trajectory (pentagram) and the operation gesture (graphic features corresponding to a static posture). Assuming the preset matching value is 0.8 (0-1 range, the closer to 1, the better the match), if the actual calculated shape matching value is 0.5 (because the pentagram drawn by the user is not regular enough and deviates significantly from the gesture features), the system will prompt "trajectory recognition does not match" through the light feedback built into the wand (red light flashes 3 times quickly). At the same time, the shape adjustment value is calculated based on the shape matching value and the preset matching value = 0.5 / 0.8 = 0.625. According to the "negative correlation adjustment" rule, the original reference confidence value of 0.7 is lowered to 0.7 × 0.625 = 0.4375, making the subsequent gesture matching judgment standard more stringent and reducing false matching.

[0088] The system acquires acceleration data from the attitude sensor in real time; for example, if the user tilts the wand upwards by 30° from a horizontal position, the Z-axis accelerometer reading will increase from 1.0 m / s². 2 Change to 3.2 m / s 2 Based on this data, the system automatically adjusts the camera module's shooting angle, causing the camera lens to rotate upwards by 30° synchronously, always maintaining correspondence with the attitude sensor's attitude and avoiding trajectory capture deviation caused by the movement of the wand. Simultaneously, the system records the most recent 5 shape matching values ​​(e.g., 0.5, 0.55, 0.6, 0.58, 0.62), calculating the average shape value as (0.5 + 0.55 + 0.6 + 0.58 + 0.62) / 5 = 0.57. According to the "shape average value negative correlation adjustment" rule, because the shape average value is low (indicating that the trajectory matching accuracy needs to be improved), the system increases the angle adjustment step of the camera module from 0.5° / time to 1° / time and the shooting frame rate from 30 frames / second to 60 frames / second, so that the camera module can adapt to the changes in the wand's posture more quickly and capture trajectory details more clearly. If the shape average value is subsequently improved to 0.85 (matching accuracy improved), the adjustment step will be reduced to 0.3° / time and the frame rate will be reduced to 20 frames / second, reducing the amount of shooting data and reducing the system's computing pressure while ensuring the matching effect.

[0089] By collecting trajectory data through the camera module and matching it with the operation trajectory, a dual verification mechanism of "trajectory + gesture" can be formed. For example, if a user wants to trigger the "flame effect," they need to complete both "drawing a circular trajectory" and "making a fist gesture" simultaneously. Only when the shape matching value of the two meets the standard can the trigger be successfully completed, effectively reducing the situation of "mistaken triggering due to similar gestures" and "successful matching despite mismatched trajectories." Dynamically adjusting the reference confidence value based on the shape matching value allows the judgment criteria for subsequent gesture matching to be flexibly adjusted according to the actual recognition effect. For example, when the deviation between trajectory and gesture is large, the judgment threshold is raised, and when the deviation is small, it is appropriately relaxed, dynamically improving the overall matching accuracy and enhancing the stability of the interaction process. Furthermore, by adjusting the camera angle in real time through acceleration data, it can be ensured that no matter how the user moves or tilts the wand, the camera module can accurately align with the posture sensor, avoiding trajectory capture omissions or deviations. Adjusting the angle, step size, and shooting frame rate based on the average shape value can improve the capture effect by increasing the frame rate and step size when the matching accuracy is insufficient, and reduce the frame rate to reduce the amount of data when the accuracy meets the standard, achieving a balance between "effect and efficiency," making trajectory interaction smoother and the system operation more efficient.

[0090] In this embodiment, the wand has four experience areas, designated A, B, C, and D. The wand has three forms: an initial form, an intermediate form, and a final form. In the initial form, the wand does not experience any interaction feedback and is in its completely primitive state. In the intermediate form, the wand traverses one or two of the areas. In the final form, the wand completes all areas, and all LEDs are illuminated.

[0091] When a wand enters an area, triggering a touch button at the area's entrance gateway resets the wand to its initial state. The wand records the reset date, and each wand can only be reset once per day. To power off, double-tap the touch button; the wand will remember its last state unless reset at the entrance gateway. Even if the user's wand is in its final form, gesture recognition is still possible. Gesture recognition can trigger success or failure lighting effects. When the gesture completion effect and feedback lighting are playing, regardless of whether other areas are activated, the wand in its final form will remain lit unless a gesture is detected.

[0092] The complete operation process of the magic wand is as follows:

[0093] 1. Press and hold for 3 seconds to turn on the device. The power-on effect is that the white LED #0 light slowly lights up within 1 second. Then, the wand body flows from LED #1 to LED #18. LED #18 performs a 1-second blue gradient, and after remaining fully lit for 1 second, all the lights gradually turn off within 1 second. The device is accompanied by vibration when powered on until the lights go out.

[0094] 2. When entering the corresponding area connected to the gateway, you need to press and hold the touch area within the corresponding area to send a signal to the gateway, and the white indicator light 1 will stay on.

[0095] 3. Press and hold the touch area, and the white LED #0 will slowly breathe, going from dark to bright and then back to dark, with a cycle of 6 seconds.

[0096] 4. When collecting gestures, LED beads numbered 17-18 will light up a light blue.

[0097] 5. The moment the gesture is completed, a meteor light effect is achieved. LEDs 2-4 light up white simultaneously and move quickly towards LED 18. After LEDs 17-18 pause, they flash brightly briefly and then turn off. A short vibration occurs during the execution of the light effect.

[0098] Feedback after the meteor light effect is completed:

[0099] A: Failed light effect, indicating insufficient gesture samples, inconsistency with the predetermined gesture, or any other unsuccessful situation. LED beads 17-18 will flash red twice quickly and then vibrate once.

[0100] B: Successful lighting effect indicates that the gesture matches the gateway's pre-defined gesture. Ions 17-18 light up a light blue, creating a breathing effect. The on / off frequency increases from 5Hz to 100Hz, then flows in the opposite direction from Ion 18 to Ion 1, filling the corresponding Ions. The lightning gesture fills Ions 2-6, the triangle gesture fills Ions 7-11, and the circle gesture fills Ions 12-16, vibrating briefly twice while executing the lighting effect.

[0101] 6. When multiple wands converge at the gateway in trigger zone D, the light gradually changes from white to light blue.

[0102] 7. The order in which the user triggers the four trigger zones (A, B, C, and D) is unpredictable. When the last trigger zone is completed, LEDs 1, 17, and 18 gradually turn blue. Then, all the lights on the wand remain constantly lit.

[0103] This embodiment also discloses a smart wand interaction system based on gyroscope gesture recognition, including a processor, wherein the processor executes the steps of the smart wand interaction method based on gyroscope gesture recognition as described in any of the above embodiments.

[0104] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A smart wand interaction method based on gyroscope gesture recognition, characterized in that, Includes the following steps: In response to the acquired start command, the gesture posture is collected in real time based on the preset posture sensor; in response to the acquired stop command, the first posture data is generated; the first posture data is converted into first gesture data using the preset first conversion algorithm; and the preset operation gesture is matched in the static database based on the first gesture data. If an operation gesture is matched, the corresponding video content and lighting content are retrieved from the preset content database. The video content is played through the preset display device, and the lighting content is projected through the preset lighting device. Otherwise, report the matching failure and wait for the startup command again; In response to the matched operation gesture, dynamic posture is collected in real time based on the posture sensor to generate second posture data. The second posture data is converted into second gesture data using a preset second conversion algorithm. Based on the second gesture data, a preset action gesture is matched in the dynamic database. If a gesture is matched, the amplitude of the gesture is calculated, and the playback speed of the video content or the projection range of the light content is adjusted according to the positive correlation of the amplitude. The first attitude data includes multiple first attitude values, and the first conversion algorithm includes the following steps: Based on the obtained abnormal threshold range, the difference between adjacent first attitude values ​​is calculated as the attitude difference. If the attitude difference is within the abnormal threshold range, the average value of the two first attitude values ​​corresponding to the attitude difference is used to replace the two first attitude values. Based on the acquisition period of the acquired gestures, obtain the length of the filter window corresponding to the acquisition period; Window filtering is performed on the first attitude data based on the filter window length; The first gesture data, which is a graphic, is identified from the first pose data according to the NNOM model; or, the direction of the action is identified from the first pose data, the direction vector is extracted according to the direction of the action, the direction vector is sorted according to the time sequence to obtain the direction vector sequence, and the first gesture data is matched from the preset graphic database according to the direction vector sequence. The step of matching preset operation gestures in the static database based on the first gesture data also includes the following sub-steps: The gesture similarity value is calculated by comparing each gesture with an element in the static database. The gesture confidence value is then calculated based on the gesture similarity value and a preset reference similarity value. The gesture confidence value with the highest value is selected as the final gesture confidence value. If the final gesture confidence value is greater than the preset reference confidence value, the corresponding element will be used as the matched operation gesture and the operation gesture will be output. The confidence ratio is calculated as the ratio of the final gesture confidence value to the reference confidence value. The length of the filter window is then adjusted based on the positive correlation of the confidence ratio. The second attitude data includes multiple second attitude values; the second conversion algorithm includes the following steps: According to the preset data length, multiple second attitude values ​​are divided into multiple attitude groups in chronological order; Calculate the average of all second attitude values ​​in each attitude group, subtract the average from all second attitude values ​​in each attitude group to obtain the second attitude dynamic value, and use the second attitude dynamic value to update the second attitude data; Calculate the unidirectional fluctuation range of the second attitude data. The unidirectional fluctuation range is the range of multiple consecutive data with the same movement trend. Calculate the number of unidirectional fluctuation intervals and the interval intervals, and calculate the average interval based on the interval intervals, where the interval interval is the time interval between unidirectional fluctuation intervals; The quantity calculation value is calculated based on the number of intervals and the preset number of interval comparisons, and the interval calculation value is calculated based on the interval interval and the preset interval comparison interval. Set the second gesture data, write the quantity calculation value and the interval calculation value into the second gesture data, and output the second gesture data.

2. The smart wand interaction method based on gyroscope gesture recognition according to claim 1, characterized in that, The step of matching preset gestures in the dynamic database based on the second gesture data also includes the following sub-steps: Extract the quantity and interval calculation values ​​from the second gesture data; If the calculated quantity is less than the preset quantity setting or the calculated interval is greater than the preset interval setting, the matching will fail. Otherwise, the matching value is calculated by weighting the quantity value and the interval value. The gesture difference is obtained by calculating the difference between each matching value and the feature value corresponding to each element in the dynamic database; the element with the smallest gesture difference is selected and the corresponding element is used as the matched action gesture and the action gesture is output. The weighted values ​​are calculated based on the positive correlation of gesture differences and the weighted values ​​based on the intervals of the negative correlation.

3. The intelligent magic wand interaction method based on gyroscope gesture recognition according to claim 2, characterized in that, The method also includes the following steps: Obtain the gesture volume library corresponding to operation gestures and action gestures; The basic operation value is matched based on the operation gesture, and the basic action value is matched based on the action gesture; Calculate the execution duration of the gesture and the duration of the action gesture; The execution weight and duration weight are calculated by normalizing the execution duration and duration. The operation volume value is calculated based on the operation base value and execution weight value. The action volume value is calculated based on the action base value and duration weight value. The final volume value is calculated based on the operation volume value and action volume value. The volume of the video content is controlled based on the final volume value.

4. The smart wand interaction method based on gyroscope gesture recognition according to claim 3, characterized in that, The method also includes the following steps: The matching ratio is calculated based on the matching calculated value and the feature value of the corresponding element; The adjustment ratio is calculated as the ratio of the confidence ratio to the matching ratio. The adjustment base value is adjusted based on the positive correlation of the adjustment ratio and the adjustment base value is adjusted based on the negative correlation of the adjustment ratio.

5. The intelligent magic wand interaction method based on gyroscope gesture recognition according to claim 4, characterized in that, The method also includes the following steps: A camera module is set up to align with the attitude sensor. Based on the start command, the camera module collects the motion trajectory of the attitude sensor to generate trajectory data. The operation trajectory is matched with the pre-set trajectory database based on the trajectory data; If an operation trajectory is matched, the stop command corresponding to the start command is obtained, and the first operation gesture after the stop command is obtained; Calculate the shape matching value between the operation trajectory and the operation gesture. If the shape matching value is less than the preset matching value, prompt that the trajectory recognition does not correspond. Calculate the shape adjustment value based on the shape matching value and the preset matching value, and adjust the reference confidence value based on the negative correlation of the shape adjustment value.

6. The intelligent magic wand interaction method based on gyroscope gesture recognition according to claim 5, characterized in that, The method also includes the following steps: The camera module acquires acceleration data from the attitude sensor in real time and adjusts the shooting angle of the camera module according to the acceleration data so that the angle of the camera module corresponds to the attitude of the attitude sensor. The average of the most recent multiple shape matching values ​​is calculated to obtain the shape average value. The angle of the camera module is adjusted, the step size and the shooting frame rate are adjusted according to the negative correlation of the shape average value.

7. A smart wand interaction system based on gyroscope gesture recognition, characterized in that, Includes a processor, wherein the steps of the smart wand interaction method based on gyroscope gesture recognition as described in any one of claims 1-6 are executed.

Images