Teaching method, device and equipment based on holographic projection and storage medium

Abstract

The invention provides a teaching method, device and equipment based on holographic projection, and a storage medium, the teaching method based on holographic projection generates holographic guidance portrait data at a corresponding target visual angle in response to a visual switching instruction of a user to a holographic imaging module, and is more beneficial for the user to observe guidance actions compared with a common video; according to the target visual angle and a preset visual angle rule, a corresponding target infrared depth sensor in the annular infrared depth sensor array is activated to collect an action data set of a user, and energy waste caused by starting of too many infrared depth sensors is avoided through targeted collection; the motion data set is bound with the visual angle ID of the target visual angle, the motion deviation is analyzed according to the visual angle ID, the holographic guidance portrait data and the motion data set, the accuracy of the motion deviation is guaranteed through binding of the visual angle ID, correction information is output according to the motion deviation, motion correction can be conducted on a user in time, and the learning effect is improved.

Description

Technical Field

[0001] This application relates to the field of education, and more particularly to a teaching method, apparatus, device, and storage medium based on holographic projection. Background Technology

[0002] Currently, the teaching of martial arts, fitness, dance and other movement-based activities mostly relies on offline classes, with coaches providing face-to-face demonstrations and guidance. However, this is limited by time, venue, and the distribution of teachers, resulting in high costs. Meanwhile, learners who practice at home mainly rely on video tutorials. The teacher's standing angle makes it impossible to fully observe the movements or receive real-time corrections, which can easily lead to improper movement habits and even sports injuries, resulting in unsatisfactory learning outcomes. Summary of the Invention

[0003] This application provides a teaching method, apparatus, device, and storage medium based on holographic projection to solve at least one problem existing in related technologies. The technical solution is as follows:

[0004] In a first aspect, embodiments of this application provide a teaching method based on holographic projection, comprising: In response to the user's visual switching command for the holographic imaging module, holographic guidance portrait data is generated at the corresponding target viewpoint; Based on the target viewpoint and preset viewpoint rules, the corresponding target infrared depth sensor in the ring infrared depth sensor array is activated to collect the user's action dataset, and the action dataset is bound to the viewpoint ID of the target viewpoint; Based on the viewpoint ID, the holographic guide portrait data, and the action dataset, the action deviation is analyzed. The action deviation includes at least one of the following: action position deviation, action amplitude deviation, and force application timing deviation. Based on the deviation in the action, output correction information.

[0005] In one implementation, the step of activating the corresponding target infrared depth sensor in the annular infrared depth sensor array to collect the user's motion dataset according to the target viewpoint and preset viewpoint rules, and binding the motion dataset with the viewpoint ID of the target viewpoint includes: The target viewpoint is matched with the preset viewpoint rules to determine the target infrared depth sensor that matches the target viewpoint in the annular infrared depth sensor array; The target infrared depth sensor is activated to collect coordinate data of each target joint point of the user, and the motion dataset corresponding to the target infrared depth sensor is determined based on the coordinate data and the timestamp of the collection. Determine the view ID of the target view and bind the action dataset to the view ID.

[0006] In one embodiment, the target infrared depth sensor includes at least two infrared depth sensors; the step of analyzing motion deviation based on the viewpoint ID, the holographic guided portrait data, and the motion dataset includes: Based on the viewpoint ID and the timestamp, target holographic data with the same target viewpoint and timestamp is determined from the holographic guidance portrait data. The target holographic data includes the standard coordinates of each target joint point of the holographic guidance portrait. Based on the coordinate data of each target joint in the motion dataset corresponding to each infrared depth sensor, determine the comprehensive coordinates of each target joint. Based on the comprehensive coordinates of each target joint and the standard coordinates of each target joint, determine the motion position deviation of each target joint and the motion amplitude deviation of at least one limb segment between the target joints. If the holographic guidance portrait data involves martial arts movements, the duration deviation of different movement stages is analyzed based on the comprehensive coordinates of each target joint point at each timestamp and the standard coordinates of each target joint point to determine the force exertion timing deviation.

[0007] In one embodiment, determining the comprehensive coordinates of each target joint point based on the coordinate data of each target joint point in the motion dataset corresponding to each infrared depth sensor includes: The azimuth angle of each infrared depth sensor and the angle difference between the target viewpoint are determined respectively, and the sensor weight of each infrared depth sensor is determined according to the angle difference. The comprehensive coordinates of each target joint are determined by weighted summation based on the coordinate data of each target joint in the motion dataset corresponding to each infrared depth sensor and the corresponding sensor weight.

[0008] In one embodiment, determining the motion position deviation of each target joint and the motion amplitude deviation of at least one limb segment between the target joints based on the composite coordinates of each target joint and the standard coordinates of each target joint includes: The composite coordinates and standard coordinates of each target joint are converted to their representations in the world coordinate system to obtain the converted composite coordinates and standard coordinates of each target joint. Based on the converted composite coordinates and converted standard coordinates under each timestamp, the distance of each target joint is determined and the average distance is calculated as the motion position deviation. At each timestamp, a standard vector of at least one limb segment between the target joints is determined based on the converted standard coordinates, and a composite vector of at least one limb segment between the target joints is determined based on the converted composite coordinates. Based on the standard vector and the composite vector, the limb rotation angle corresponding to each limb segment under each timestamp is determined, and the movement amplitude deviation of each limb segment is determined based on the limb rotation angle under each timestamp and the preset limb rotation angle corresponding to each limb segment; wherein, the movement amplitude deviation includes roll angle deviation, pitch angle deviation and yaw angle deviation.

[0009] In one implementation, the step of analyzing the duration deviation of different action stages to determine the force exertion timing deviation based on the comprehensive coordinates of each of the target joints at each timestamp and the standard coordinates of each of the target joints includes: Based on the holographic guided portrait data, the target key points of interest are determined, and based on the comprehensive coordinates of the target key points of interest, the actual movement speed and actual acceleration of the target key points of interest at each timestamp are determined. The time stamp when the actual acceleration first exceeds the acceleration threshold is taken as the first dividing point between the power accumulation action phase and the power exertion action phase, and the time stamp when the actual speed of motion drops from the maximum to a preset percentage is taken as the second dividing point between the power exertion action phase and the power recovery action phase. Based on the first and second dividing points, the actual durations corresponding to the power-accelerating action phase, the power-generating action phase, and the power-recovering action phase are determined respectively. Based on the standard coordinates of each target joint at each timestamp, the standard motion speed and standard acceleration of the target joint of interest are determined to determine the standard durations corresponding to the power-accelerating action phase, the power-generating action phase, and the power-recovering action phase. Based on all the actual movement speeds and all the standard movement speeds, determine the speed similarity. Based on the actual and standard durations of the power-accumulating phase, the actual and standard durations of the power-generating phase, and the actual and standard durations of the finishing phase, determine the duration deviations of the three phases respectively. Based on each duration deviation and the speed similarity, perform a weighted calculation to determine the power-generating timing deviation.

[0010] In one implementation, outputting correction information based on the action deviation includes: The movement position deviation is compared with a first deviation threshold and the movement amplitude deviation is compared with a second deviation threshold. When the movement position deviation is greater than the first deviation threshold, the movement position correction information is output. When the movement amplitude deviation is greater than the second deviation threshold, the movement amplitude correction information is output. or, The average motion position deviation is calculated and normalized based on the motion position deviation of each target joint. The average motion amplitude deviation is calculated and normalized based on the motion amplitude deviation of all limb segments. The comprehensive deviation is determined by weighted calculation based on the normalized average motion position deviation, motion position weight parameter, normalized average motion amplitude deviation, and motion amplitude weight. If the comprehensive deviation is greater than the third deviation threshold, comprehensive correction information including at least one of motion position and motion amplitude is output. The deviation in the timing of force application is compared with the fourth deviation threshold. If the deviation in the timing of force application is greater than the fourth deviation threshold, the correction information for the timing of force application is output.

[0011] Secondly, embodiments of this application provide a teaching device based on holographic projection, comprising: The response module is used to respond to the user's visual switching command to the holographic imaging module and generate holographic guidance portrait data at the corresponding target viewpoint; The acquisition module is used to activate the corresponding target infrared depth sensor in the ring infrared depth sensor array to acquire the user's action dataset according to the target viewpoint and preset viewpoint rules, and bind the action dataset to the viewpoint ID of the target viewpoint; The analysis module is used to analyze motion deviations based on the viewpoint ID, the holographic guide portrait data, and the motion dataset. The motion deviations include at least one of motion position deviation, motion amplitude deviation, and force application timing deviation. The correction module is used to output correction information based on the deviation of the action.

[0012] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the methods in any of the above-described embodiments.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed, implements the methods in any of the above-described embodiments.

[0014] The beneficial effects of the above technical solution include at least the following: By responding to the user's visual switching command to the holographic imaging module, holographic guidance portrait data is generated at the corresponding target viewpoint, which is more conducive to user observation of guided actions compared to ordinary video. According to the target viewpoint and preset viewpoint rules, the corresponding target infrared depth sensor in the ring infrared depth sensor array is activated to collect the user's action dataset, which is targeted to avoid energy waste caused by activating too many infrared depth sensors. The action dataset is bound to the viewpoint ID of the target viewpoint. Based on the viewpoint ID, holographic guidance portrait data, and action dataset, action deviations are analyzed. The accuracy of action deviations is ensured by binding the viewpoint ID. Based on the action deviations, correction information is output, which is conducive to timely correction of user actions and improves learning effectiveness.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, these aspects, embodiments, and features will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0016] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0017] Figure 1 This is a flowchart illustrating the steps of a teaching method based on holographic projection according to an embodiment of this application; Figure 2 This is a structural block diagram of a holographic projection-based teaching device according to an embodiment of this application; Figure 3 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0019] Reference Figure 1 The flowchart illustrates a teaching method based on holographic projection according to an embodiment of this application. This teaching method based on holographic projection may include at least steps S100-S400: S100: In response to the user's visual switching command for the holographic imaging module, holographic guidance portrait data is generated at the corresponding target viewpoint.

[0020] S200. Based on the target viewpoint and preset viewpoint rules, activate the corresponding target infrared depth sensor in the ring infrared depth sensor array to collect the user's action dataset, and bind the action dataset with the viewpoint ID of the target viewpoint.

[0021] S300 analyzes motion deviations based on viewpoint ID, holographic guided portrait data, and motion dataset.

[0022] Optionally, the motion deviation includes motion position deviation, motion amplitude deviation, and force application timing deviation. Other embodiments may include one or more of motion position deviation, motion amplitude deviation, and force application timing deviation.

[0023] S400: Output correction information based on motion deviation.

[0024] The technical solution of this application embodiment generates holographic guidance portrait data at the corresponding target viewpoint in response to the user's visual switching command to the holographic imaging module. This is more conducive to the user's observation of the guidance actions compared to ordinary videos. According to the target viewpoint and preset viewpoint rules, the corresponding target infrared depth sensor in the ring infrared depth sensor array is activated to collect the user's action dataset. Targeted collection avoids energy waste caused by activating too many infrared depth sensors. The action dataset is bound to the viewpoint ID of the target viewpoint. Based on the viewpoint ID, holographic guidance portrait data, and action dataset, action deviations are analyzed. The accuracy of action deviations is ensured by binding the viewpoint ID. Based on the action deviations, correction information is output, which is conducive to timely correction of the user's actions and improves the learning effect.

[0025] Reference Figure 2 The diagram shows a structural block diagram of a holographic projection-based teaching device according to an embodiment of this application. The device may include: The response module is used to respond to the user's visual switching command to the holographic imaging module and generate holographic guidance portrait data at the corresponding target viewpoint; The acquisition module is used to activate the corresponding target infrared depth sensor in the ring infrared depth sensor array to acquire the user's action dataset according to the target viewpoint and preset viewpoint rules, and bind the action dataset with the viewpoint ID of the target viewpoint. The analysis module is used to analyze motion deviations based on the viewpoint ID, holographic guided human image data, and motion dataset. Motion deviations include at least one of motion position deviation, motion amplitude deviation, and force application timing deviation. The correction module is used to output correction information based on the deviation of the action.

[0026] The functions of each module in the device of this application embodiment can be found in the corresponding description in the above method, and will not be repeated here.

[0027] In one embodiment, the response module, acquisition module, analysis module, and correction module of this application are all part of the processing system, used to respond to the user's visual switching command to the holographic imaging module and control the holographic imaging module to generate holographic guidance portrait data at the corresponding target viewpoint. For example, the device of this application has a housing, and a 360° holographic imaging module is installed inside the housing, along with a ring-shaped infrared depth sensor array. Each infrared depth sensor corresponds to an azimuth angle (which can cover a certain range centered on the azimuth angle, such as 60°). By overlapping the viewpoints to cover the 360° viewpoint, the user can select the action to be learned (such as martial arts) from the holographic action model library of the processing system, such as the holographic imaging module, through voice, gestures, buttons, etc., and issue a visual switching command, such as a 45° visual switching command. Then, the 45° is used as the target viewpoint, and holographic guidance portrait data for martial arts is generated at 45°. If it is 90°, then holographic guidance portrait data for martial arts is generated at 90°.

[0028] Optionally, the holographic imaging module in this embodiment incorporates a negative refraction flat lens and a high-brightness LED direct light source. The light source emits vertically upwards, and after refraction by the lens, a 1:1 life-size stereoscopic holographic image is formed at a height of 1.5-2.0m above the device. The imaging unit has a built-in offline holographic motion model library, which can pre-store standardized real-person holographic guidance image data for martial arts movements, dance movements, etc., supporting one-click retrieval and loop playback. The image viewing angle can be adjusted 360° to meet the needs of learners to observe movement details from different angles. Furthermore, the holographic guidance supports the acquisition and import of real-person motion images. Instructors can record 1080P high-definition motion videos (supporting multi-angle recording) through the accompanying terminal APP. After automatic processing by the device's built-in 3D modeling algorithm, the videos are converted into 1:1 life-size holographic guidance image data. The images retain the instructor's limb shape, movement trajectory, force details, and other realistic features, supporting loop playback and slow-motion breakdown playback to meet the needs of standardized martial arts movement teaching. In addition, the processing system has a built-in motion correction algorithm model to analyze and determine motion deviations and output correction information.

[0029] In this embodiment, to collect data in a targeted manner, save energy, and reduce data processing load, preset viewing angle rules are configured in advance. For example, a viewing angle ID is set for each angle range. For instance, a viewing angle ID can be set for 0~45°, corresponding to infrared depth sensors numbered 0 and 1 (this embodiment takes at least two infrared depth sensors for each viewing angle as an example; other embodiments may include only one). A viewing angle ID is set for 45~90°, corresponding to infrared depth sensors numbered 2 and 3, and so on, and mapped and calibrated with the viewing angle of the holographic imaging module. Therefore, on the one hand, it ensures that the corresponding infrared depth sensor is activated at a certain viewing angle while others are not activated, achieving the purpose of energy saving; on the other hand, for example, at 45°, since the holographic guidance portrait data is generated at 45°, the user's actions at 45° are also collected based on the corresponding infrared depth sensor. Only then is the analysis of subsequent action deviations comparable, improving accuracy.

[0030] In one implementation, step S200 includes steps S210-S230: S210. Match the target viewpoint with the preset viewpoint rules to determine the target infrared depth sensor that matches the target viewpoint in the annular infrared depth sensor array.

[0031] Optionally, the target viewing angle is matched with a preset viewing angle rule. For example, if the target viewing angle is 45°, the target infrared depth sensor corresponding to the target viewing angle in the ring infrared depth sensor array is determined to be the infrared depth sensor numbered 1 and 2, that is, the infrared depth sensor numbered 1 and 2 is the target infrared depth sensor.

[0032] S220. Activate the target infrared depth sensor, collect the coordinate data of each target joint point of the user, and determine the motion dataset corresponding to the target infrared depth sensor based on the coordinate data and the timestamp of the collection.

[0033] Optionally, the control system activates the target infrared depth sensor within 10ms, completing the activation and sleep switching of the infrared depth sensor to ensure the synchronization of viewpoint switching and data acquisition. Then, based on a sampling frequency of 60fps, it collects the coordinate data (3D) of each target joint point of the user in real time through the target infrared depth sensor, and records the timestamp of the coordinate data acquisition. Based on the coordinate data and timestamp, the motion dataset corresponding to the target infrared depth sensor is determined. It should be noted that the target joint points can be preset or configured as target joint points based on movements (such as martial arts and dance), and are not limited to shoulder, elbow, wrist, hip, knee, ankle, waist, etc. Therefore, if two infrared depth sensors are activated, each infrared depth sensor will have a motion dataset for the target joint point.

[0034] In this embodiment, the processing system establishes a verification log to record the activation time and data acquisition duration of the infrared depth sensor for each viewpoint switch, verifying whether the synchronization meets the standard, which facilitates subsequent adjustments. For example, each time the synchronization delay time ΔT_synchronization = |T_viewpoint switch completion - T_activation completion| is determined, and it is checked whether it meets the standard of ≤10ms. If it does, the sampled data is considered valid; otherwise, the data set is discarded and re-acquired.

[0035] In some embodiments, the annular infrared depth sensor array may also be configured as a rotatable structure.

[0036] S230. Determine the view ID of the target view and bind the action dataset to the view ID.

[0037] Optionally, since users may issue multiple visual switching commands during the learning process, for example, users may want to observe the action from different angles. Therefore, there will be a corresponding action dataset for the same target joint. So, view IDs for all viewpoints are set in advance. Then, based on the view ID of the target viewpoint, the view ID is bound to the action dataset and the corresponding holographic guide portrait data of the target viewpoint. Then, in subsequent comparisons, the action dataset with the same view ID can be compared with the holographic guide portrait data with the same view ID.

[0038] In one implementation, step S300 includes steps S310-S340: S310. Based on the viewpoint ID and timestamp, determine the target holographic data with the same target viewpoint and timestamp from the holographic guide portrait data.

[0039] Optionally, based on the viewpoint ID and timestamp, target holographic data with the same target viewpoint and timestamp are determined from the holographic guide portrait data. The target holographic data includes the standard coordinates (three-dimensional) of each target joint point of the holographic guide portrait.

[0040] S320. Determine the comprehensive coordinates of each target joint based on the coordinate data of each target joint in the motion dataset corresponding to each infrared depth sensor.

[0041] Specifically: 1. Determine the azimuth angle of each infrared depth sensor and the angle difference between the target viewpoint, and determine the sensor weight of each infrared depth sensor based on the angle difference.

[0042] It should be noted that the azimuth angle of each infrared depth sensor is calculated separately from the angle difference between the target's viewing angle, and then the sensor weight of each infrared depth sensor is determined based on the angle difference. For example, suppose there are a total of 8 sensors with an azimuth angle of θ._i = i×45° (i=0,1,...,7), where i is the number. Calculate the angle difference between the azimuth angle of each infrared depth sensor and the target viewing angle θ, such as 45°: Δθ_i = min(|θ-θ_i|,360°-|θ-θ_i|). Then set the weight of the infrared depth sensor using the formula: w_i = max(0,1 -Δθ_i / 90°).

[0043] 2. Based on the coordinate data of each target joint in the motion dataset corresponding to each infrared depth sensor and the corresponding sensor weight, perform a weighted summation to determine the comprehensive coordinates of each target joint.

[0044] For example, if there are two infrared depth sensors, then for each target joint point in each time stamp, the comprehensive coordinates of each target joint point in each time stamp are determined by weighted summation based on the coordinate data of one infrared depth sensor and the sensor weight of that infrared depth sensor, the coordinate data of the other infrared depth sensor and the sensor weight of the other infrared depth sensor.

[0045] Therefore, the system ultimately achieves synchronous dynamic sampling that is "what you see is what you get". When the user switches perspectives, the system automatically optimizes the sensor acquisition weights for the corresponding orientation to ensure that the acquired data is highly consistent with the observation perspective.

[0046] S330. Based on the comprehensive coordinates of each target joint and the standard coordinates of each target joint, determine the motion position deviation of each target joint and the motion amplitude deviation of at least one limb segment between the target joints.

[0047] Specifically: 1. Convert the composite coordinates and standard coordinates of each target joint into the world coordinate system to obtain the converted composite coordinates and standard coordinates of each target joint.

[0048] 2. Based on the converted composite coordinates and converted standard coordinates under each timestamp, determine the distance of each target joint and calculate the average distance as the motion position deviation.

[0049] For example, the transformed composite coordinates of a target key point at a certain timestamp are: And the transformed standard coordinates are Then calculate the distance. So, for a given target node, the distance of all timestamps... The average distance can be obtained by adding the values ​​together. Therefore, based on this method, the average distance of each target joint can be calculated as the corresponding motion position deviation.

[0050]

[0051] 3. At each timestamp, determine the standard vector of at least one limb segment between the target joints based on the converted standard coordinates, and determine the composite vector of at least one limb segment between the target joints based on the converted composite coordinates.

[0052] It should be noted that the limb segments are also determined based on the learned movements. For example, in the "horse stance twisting waist" movement, the standard vector of the limb segment between the two target joints, hip and waist, is determined based on the transformed standard coordinates A and B. =BA (Some movements can identify more than two limb segments; this is just an example and not a limitation). Similarly, the comprehensive vector of the limb segments between the target joints is determined based on the comprehensive coordinates after the transformation of the two target joints, hip and waist. .

[0053] 4. Based on the standard vector and the composite vector, determine the limb rotation angle corresponding to each limb segment at each time stamp, and determine the movement amplitude deviation of each limb segment based on the limb rotation angle at each time stamp and the preset limb rotation angle corresponding to each limb segment.

[0054] In this embodiment of the application, based on standard vectors and composite vector Construct the formula: The rigid body rotation matrix is ​​solved by matrix inverse operation. This allows us to obtain the limb rotation angle corresponding to each limb segment at each time stamp, including the roll angle. Pitch angle and yaw angle Specifically, the standard vector is compared with the reference vector in the world coordinate system (such as the vertical and horizontal directions). Through the inverse operation of the rigid body rotation matrix, the three Euler angles (α, β, γ) that allow the reference vector to rotate to the target direction vector are derived, thereby determining the rigid body rotation matrix. :

[0055] Then, based on the preset limb rotation angles (i.e., standard roll angle, standard pitch angle, and standard yaw angle) corresponding to each timestamp, respectively, and the roll angle... Pitch angle Yaw angle The absolute value of the angle difference is calculated to obtain three absolute values ​​corresponding to each timestamp. Then, the three absolute values ​​of all timestamps are averaged to obtain the roll angle deviation, pitch angle deviation, and yaw angle deviation, which constitute the motion amplitude deviation.

[0056] S340. If the holographic guidance image data involves martial arts movements, analyze the duration deviation of different movement stages based on the comprehensive coordinates of each target joint point at each time stamp and the standard coordinates of each target joint point to determine the force exertion timing deviation.

[0057] Specifically: 1. Based on holographic guided human portrait data, determine the target key points of interest, and based on the comprehensive coordinates of the target key points of interest, determine the actual movement speed and actual acceleration of the target key points of interest at each time stamp.

[0058] It should be noted that the target joints to be focused on are pre-defined in the holographic motion model library based on different movements. For example, in a martial arts movement like "horse stance with waist twist," the target joints to be focused on are the shoulder and hip. Other martial arts movements can use other joints, and the number of joints can be adjusted adaptively without specific limitations. Therefore, based on the specific movements corresponding to the holographic guided portrait data, the target joints to be focused on can be determined through the holographic motion model library. Then, based on the comprehensive coordinates of the target joints to be focused on, the actual motion velocity v(t) (v(t) = L(t) / Δt) and the actual acceleration a(t) (a(t) = [v(t+Δt) - v(t)] / Δt) of the target joints to be focused on at each time point are calculated and determined; where t is the time point of the time point, Δt = 1 / 60 second (based on a 60Hz sampling frequency), L(t) is the distance between the comprehensive coordinates of adjacent time points, and [v(t+Δt) - v(t)] is the difference between the actual motion velocities of adjacent time points.

[0059] 2. The time stamp when the actual acceleration first exceeds the acceleration threshold (based on the amplitude of the movement, such as 0.5, 2 m / s², etc.) is taken as the first dividing point between the power-accelerating action phase (the muscle pre-tension phase from the start of the movement to the instant before the exertion) and the power-exerting action phase (the phase of maximum acceleration during the main force output). The time stamp when the actual speed decreases from the maximum (i.e. the maximum actual speed among all the actual speeds determined by the time stamps) to a preset percentage (e.g., 70%) is taken as the second dividing point between the power-exerting action phase and the recovery action phase (the braking and balancing phase from the end of the exertion to the finalization of the movement).

[0060] 3. Based on the first and second dividing points, determine the actual duration of the power-accelerating, power-generating, and power-recovering action phases, respectively. Based on the standard coordinates of each target joint at each time stamp, determine the standard motion speed and standard acceleration of the target joints of interest to determine the standard duration of the power-accelerating, power-generating, and power-recovering action phases.

[0061] Specifically, because the first and second boundary points are determined, the actual durations of the charging, exerting, and finishing action phases can be determined separately, i.e., which duration belongs to which action phase. Then, based on the standard coordinates of each target joint at each time stamp, the standard motion velocity and standard acceleration of the target joint of interest are determined to determine the standard durations of the charging, exerting, and finishing action phases. This can be determined based on the same principle mentioned above, or configured in advance in the holographic motion model library.

[0062] 4. Based on all actual movement speeds and all standard movement speeds, determine the speed similarity. Based on the actual and standard durations of the power-accumulating phase, the power-generating phase, and the recovery phase, determine the duration deviations of the three phases respectively. Then, perform a weighted calculation based on each duration deviation and the speed similarity to determine the power-generating timing deviation.

[0063] Specifically, based on all actual and standard motion speeds, speed similarity is determined. For example, actual and standard motion speed curves can be generated separately, and then the similarity between the two speed curves can be calculated using the Pearson correlation coefficient. It should be noted that if there are multiple target joints, the same operation can be performed on each one to determine the actual and standard motion speed curves, as well as the similarity r between them. Next, based on the actual and standard durations of the power-gathering phase, the power-exerting phase, and the recovery phase, the duration deviations of the three phases are determined. Specifically, the duration deviation is obtained by subtracting the absolute value of the actual and standard durations of different phases. Finally, a weighted calculation is performed based on the duration deviations and speed similarities to determine the timing deviation for force application. Specifically, the average duration deviation of each target joint is calculated for all interested target joints to obtain the average duration deviation ΔT'. Then, if there are multiple target joints, the average speed similarity r' is calculated based on the speed similarities. If there is only one target joint, it is used as the average speed similarity. Finally, the timing deviation S_timing is calculated based on the formula: S_timing = w_T×exp(-ΔT' / T0) + w_r'×max(0, r') Where w_T is the weighting coefficient corresponding to the average duration deviation, w_r' is the weighting coefficient corresponding to the average speed similarity, and T0 is the preset time constant.

[0064] In one implementation, step S400 may include the following steps (in no particular order): S410. Compare the motion position deviation with the first deviation threshold and the motion amplitude deviation with the second deviation threshold respectively. When the motion position deviation is greater than the first deviation threshold, output the motion position correction information. When the motion amplitude deviation is greater than the second deviation threshold, output the motion amplitude correction information.

[0065] Optionally, a first deviation threshold, such as 1.5cm or 2cm, and a second deviation threshold, such as 5° or 8°, can be preset and adjusted based on actual conditions. The movement position deviation is compared with the first deviation threshold, and the movement amplitude deviation is compared with the second deviation threshold. When the movement position deviation is greater than the first deviation threshold, it indicates that the deviation is too high, and the correction information for the movement position is output, such as determining the distance to the target joint. The maximum preset number of target joints is set, and correction information is output for each of these target joints. In addition, when the deviation of the motion amplitude is greater than the second deviation threshold, correction information for the motion amplitude is output. For example, if any one of the roll angle deviation, pitch angle deviation, and yaw angle deviation is greater than the second deviation threshold, then correction information for the corresponding limb segment is output for the specific deviation that is greater than the second deviation threshold.

[0066] It should be noted that, referring to medical data on adult joint range of motion and professional standards for martial arts movements (such as the physiological limit of the adult lumbar roll angle being approximately ±45°, and the effective training range for the martial arts "twisting waist" movement being +15° to ±30°), if the calculated lumbar roll angle is ±50°, it is determined to be sensor noise, and the data is discarded and resampled. If the degree to which the movement exceeds the effective training range within the normal range is higher than the second deviation threshold (e.g., the standard requirement is +20°, and the student's is ±35°), then "movement over-amplitude" is determined, and correction information is subsequently provided. This embodiment of the application improves the accuracy of angle calculation and avoids misjudgment in teaching caused by abnormal data, ensuring that the algorithm output fits the actual needs of adult martial arts teaching.

[0067] S420. Calculate and normalize the average motion position deviation based on the motion position deviation of each target joint. Calculate and normalize the average motion amplitude deviation based on the motion amplitude deviation of all limb segments. Perform weighted calculation based on the normalized average motion position deviation, motion position weight parameter, normalized average motion amplitude deviation, and motion amplitude weight to determine the comprehensive deviation. If the comprehensive deviation is greater than the third deviation threshold, output comprehensive correction information including at least one of motion position and motion amplitude.

[0068] It should be noted that, in addition to the S410 method, the average motion position deviation can also be calculated and normalized based on the motion position deviation of each target joint (e.g., each motion position deviation is divided by the largest motion position deviation, i.e., the maximum distance corresponding to the target joint, or the maximum distance ΔP_max is directly preset to 10cm). The average motion amplitude deviation can also be calculated and normalized based on the motion amplitude deviation of all limb segments (e.g., roll angle deviation, pitch angle deviation, and yaw angle deviation are all divided by the roll angle). Pitch angle Yaw angle The maximum absolute value of the angle difference is calculated. Then, a weighted calculation is performed based on the normalized average motion position deviation, motion position weight parameter, normalized average motion amplitude deviation, and motion amplitude weight (i.e., the weight terms of motion position and motion amplitude are calculated separately and then added together) to determine the comprehensive deviation, rather than judging them separately.

[0069] Specifically, the third deviation threshold can be set to, for example, 0.6. A deviation between 0.4 and 0.6 is considered good, less than 0.4 is considered excellent, and greater than 0.6 indicates a need for improvement. Therefore, if the overall deviation exceeds the third deviation threshold, the output includes comprehensive correction information for at least one of the movement position and movement amplitude. For example, it can be analyzed which has a lower weight: the weight of the movement position or the weight of the movement amplitude. If they are the same, the corresponding correction information for both is output; otherwise, the correction information for the lower weight is output.

[0070] S430. Compare the force application timing deviation with the fourth deviation threshold. If the force application timing deviation is greater than the fourth deviation threshold, output the force application timing correction information.

[0071] Specifically, the timing deviation of force application is compared with the fourth deviation threshold. If the timing deviation is greater than the fourth deviation threshold, it indicates a large timing deviation. In this case, corrective information on the timing of force application is output, such as suggestions on the duration of different action phases and speed control. For example, if it is determined that the timing of force application is too late or too early.

[0072] In some embodiments, a speed threshold can be set. If the maximum actual speed during the power-accumulating action phase exceeds the speed threshold, the rate is determined to be too fast and is judged as "insufficient power accumulation". Alternatively, the average speed change rate of different action phases can be analyzed. If the average speed change rate is less than the change rate threshold (e.g., 15%), it is considered qualified. Otherwise, the corresponding power-generating timing correction information is output.

[0073] In some implementations, the target key points involved in the correction information can be highlighted in the holographic guidance portrait data (e.g., marked with red to visually show the direction of correction).

[0074] In this embodiment, holographic guidance portrait data is generated through a holographic imaging module, and different perspectives can be switched based on user needs, realizing a wearable-free action teaching experience of "learning immediately upon startup". It provides users with all-round visual learning, and the user's action standardization assessment dimension is multi-dimensional, which facilitates comprehensive, accurate and efficient learning for users. Compared with single vision or single sensor, the teaching efficiency of complex actions is improved by ≥35%. Furthermore, by switching different perspectives, a 360° comprehensive action dataset can be obtained, such as the kick height when viewed from the side and the waist twisting amplitude when viewed from behind, which can analyze action deviations in the most comprehensive and accurate way.

[0075] Through the embodiments of this application, 1. it can break through the limitations of teaching time and space and improve the convenience of learning: the holographic imaging module using media-free aerial imaging technology can generate 1:1 life-size three-dimensional images of martial arts / dance movements in any indoor scene such as home or small training institution, without the need for professional venues and on-site coaching; the built-in offline movement database supports offline use, and the independently powered battery provides a battery life of ≥4 hours on a single charge, completely solving the problems of traditional offline teaching being constrained by time, location, and teacher distribution, allowing learners to carry out standardized movement practice anytime, anywhere; 2. Real-life three-dimensional demonstration + precise error correction ensures the standardization of movement learning: the generated real-life three-dimensional holographic images support 360° viewing angle adjustment, allowing learners to clearly observe the force trajectory, limb spatial orientation, and other details of martial arts moves, avoiding movement cognition deviations caused by the limitations of two-dimensional perspective; combined with a 60 frames / second infrared motion capture and error correction algorithm model, it can identify movement position deviations, movement amplitude deviations, and force timing deviations in real time, and guide learning through a dual feedback form of voice broadcast + image highlighting. 1. Timely correction of movements effectively avoids sports injuries caused by incorrect practice, significantly improving the efficiency and standardization of movement learning; 2. Miniaturized integrated design lowers the barrier to entry. The overall base of the device occupies ≤0.3㎡, adopting an integrated structure and a simple touch operation panel, eliminating the need for complicated installation and debugging processes, making it suitable for learners of different ages; Bluetooth and WiFi interfaces are reserved, supporting interconnection with mobile phones, tablets, and other terminals, enabling the uploading and updating of personalized live-action movement courses, catering to the needs of individual home learning and centralized teaching in small training institutions, significantly reducing the application threshold and usage cost of intelligent sports and arts teaching equipment; 3. Expanding teaching scenarios and possessing broad promotional value: It is not only suitable for teaching movement-based sports and arts such as martial arts and dance, but can also be extended to fitness shaping, opera movements, and other fields that require precise movement demonstrations, possessing strong functional compatibility and scenario adaptability; Pure optical imaging has no media loss, low equipment maintenance costs, and is easy to promote and apply on a large scale, which is of great significance for promoting the popularization of intelligent education equipment and the innovation of movement teaching models.

[0076] Reference Figure 3The diagram illustrates a structural block diagram of an electronic device according to an embodiment of this application. The electronic device includes a memory 310 and a processor 320. The memory 310 stores instructions that can be executed on the processor 320. The processor 320 loads and executes these instructions to implement the holographic projection-based teaching method described in the above embodiment. The number of memories 310 and processors 320 can be one or more.

[0077] In one embodiment, the electronic device further includes a communication interface 330 for communicating with external devices and exchanging data. If the memory 310, processor 320, and communication interface 330 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0078] Optionally, in a specific implementation, if the memory 310, processor 320 and communication interface 330 are integrated on a single chip, the memory 310, processor 320 and communication interface 330 can communicate with each other through an internal interface.

[0079] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the holographic projection-based teaching method provided in the above embodiments.

[0080] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this application.

[0081] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0082] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0083] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0084] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0085] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0087] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0088] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0089] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0090] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0091] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A teaching method based on holographic projection, characterized in that, include: In response to the user's visual switching command for the holographic imaging module, holographic guidance portrait data is generated at the corresponding target viewpoint; Based on the target viewpoint and preset viewpoint rules, the corresponding target infrared depth sensor in the ring infrared depth sensor array is activated to collect the user's action dataset, and the action dataset is bound to the viewpoint ID of the target viewpoint; Based on the viewpoint ID, the holographic guide portrait data, and the action dataset, the action deviation is analyzed. The action deviation includes at least one of the following: action position deviation, action amplitude deviation, and force application timing deviation. Based on the deviation in the action, output correction information.

2. The teaching method based on holographic projection according to claim 1, characterized in that: The step of activating the corresponding target infrared depth sensor in the annular infrared depth sensor array to collect the user's motion dataset according to the target viewpoint and preset viewpoint rules, and binding the motion dataset with the viewpoint ID of the target viewpoint includes: The target viewpoint is matched with the preset viewpoint rules to determine the target infrared depth sensor that matches the target viewpoint in the annular infrared depth sensor array; The target infrared depth sensor is activated to collect coordinate data of each target joint point of the user, and the motion dataset corresponding to the target infrared depth sensor is determined based on the coordinate data and the timestamp of the collection. Determine the view ID of the target view and bind the action dataset to the view ID.

3. The teaching method based on holographic projection according to claim 2, characterized in that: The target infrared depth sensor includes at least two infrared depth sensors; the analysis of motion deviation based on the viewpoint ID, the holographic guided portrait data, and the motion dataset includes: Based on the viewpoint ID and the timestamp, target holographic data with the same target viewpoint and timestamp is determined from the holographic guidance portrait data. The target holographic data includes the standard coordinates of each target joint point of the holographic guidance portrait. Based on the coordinate data of each target joint in the motion dataset corresponding to each infrared depth sensor, determine the comprehensive coordinates of each target joint. Based on the comprehensive coordinates of each target joint and the standard coordinates of each target joint, determine the motion position deviation of each target joint and the motion amplitude deviation of at least one limb segment between the target joints. If the holographic guidance portrait data involves martial arts movements, the duration deviation of different movement stages is analyzed based on the comprehensive coordinates of each target joint point at each timestamp and the standard coordinates of each target joint point to determine the force exertion timing deviation.

4. The teaching method based on holographic projection according to claim 3, characterized in that: The step of determining the comprehensive coordinates of each target joint point based on the coordinate data of each target joint point in the motion dataset corresponding to each infrared depth sensor includes: The azimuth angle of each infrared depth sensor and the angle difference between the target viewpoint are determined respectively, and the sensor weight of each infrared depth sensor is determined according to the angle difference. The comprehensive coordinates of each target joint are determined by weighted summation based on the coordinate data of each target joint in the motion dataset corresponding to each infrared depth sensor and the corresponding sensor weight.

5. The teaching method based on holographic projection according to claim 3, characterized in that: The step of determining the motion position deviation of each target joint and the motion amplitude deviation of at least one limb segment between the target joints based on the comprehensive coordinates and the standard coordinates of each target joint includes: The composite coordinates and standard coordinates of each target joint are converted to their representations in the world coordinate system to obtain the converted composite coordinates and standard coordinates of each target joint. Based on the converted composite coordinates and converted standard coordinates under each timestamp, the distance of each target joint is determined and the average distance is calculated as the motion position deviation. At each timestamp, a standard vector of at least one limb segment between the target joints is determined based on the converted standard coordinates, and a composite vector of at least one limb segment between the target joints is determined based on the converted composite coordinates. Based on the standard vector and the composite vector, the limb rotation angle corresponding to each limb segment under each timestamp is determined, and the movement amplitude deviation of each limb segment is determined based on the limb rotation angle under each timestamp and the preset limb rotation angle corresponding to each limb segment; wherein, the movement amplitude deviation includes roll angle deviation, pitch angle deviation and yaw angle deviation.

6. The teaching method based on holographic projection according to claim 5, characterized in that: The step of analyzing the duration deviation of different action stages to determine the force exertion timing deviation based on the comprehensive coordinates of each target joint at each timestamp and the standard coordinates of each target joint includes: Based on the holographic guided portrait data, the target key points of interest are determined, and based on the comprehensive coordinates of the target key points of interest, the actual movement speed and actual acceleration of the target key points of interest at each timestamp are determined. The time stamp when the actual acceleration first exceeds the acceleration threshold is taken as the first dividing point between the power accumulation action phase and the power exertion action phase, and the time stamp when the actual speed of motion drops from the maximum to a preset percentage is taken as the second dividing point between the power exertion action phase and the power recovery action phase. Based on the first and second dividing points, the actual durations corresponding to the power-accelerating action phase, the power-generating action phase, and the power-recovering action phase are determined respectively. Based on the standard coordinates of each target joint at each timestamp, the standard motion speed and standard acceleration of the target joint of interest are determined to determine the standard durations corresponding to the power-accelerating action phase, the power-generating action phase, and the power-recovering action phase. Based on all the actual movement speeds and all the standard movement speeds, determine the speed similarity. Based on the actual and standard durations of the power-accumulating phase, the actual and standard durations of the power-generating phase, and the actual and standard durations of the finishing phase, determine the duration deviations of the three phases respectively. Based on each duration deviation and the speed similarity, perform a weighted calculation to determine the power-generating timing deviation.

7. The teaching method based on holographic projection according to claim 3, characterized in that: The step of outputting correction information based on the action deviation includes: The movement position deviation is compared with a first deviation threshold and the movement amplitude deviation is compared with a second deviation threshold. When the movement position deviation is greater than the first deviation threshold, the movement position correction information is output. When the movement amplitude deviation is greater than the second deviation threshold, the movement amplitude correction information is output. or, The average motion position deviation is calculated and normalized based on the motion position deviation of each target joint. The average motion amplitude deviation is calculated and normalized based on the motion amplitude deviation of all limb segments. The comprehensive deviation is determined by weighted calculation based on the normalized average motion position deviation, motion position weight parameter, normalized average motion amplitude deviation, and motion amplitude weight. If the comprehensive deviation is greater than the third deviation threshold, comprehensive correction information including at least one of motion position and motion amplitude is output. The deviation in the timing of force application is compared with the fourth deviation threshold. If the deviation in the timing of force application is greater than the fourth deviation threshold, the correction information for the timing of force application is output.

8. A teaching device based on holographic projection, characterized in that, include: The response module is used to respond to the user's visual switching command to the holographic imaging module and generate holographic guidance portrait data at the corresponding target viewpoint; The acquisition module is used to activate the corresponding target infrared depth sensor in the ring infrared depth sensor array to acquire the user's action dataset according to the target viewpoint and preset viewpoint rules, and bind the action dataset to the viewpoint ID of the target viewpoint; The analysis module is used to analyze motion deviations based on the viewpoint ID, the holographic guide portrait data, and the motion dataset. The motion deviations include at least one of motion position deviation, motion amplitude deviation, and force application timing deviation. The correction module is used to output correction information based on the deviation of the action.

9. An electronic device, characterized in that, include: A processor and a memory, wherein instructions are stored in the memory and loaded and executed by the processor to implement the method as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program therein, which, when executed, implements the method as described in any one of claims 1-7.

Images