A white edge coordinate system for a camer module

Abstract

The present invention relates to the technical field of motion capture, and discloses a white edge coordinate system for a camer module, which includes an inertial sensor module, an action database module, a human body action model establishment module, an action capture module, a data reading module and an action reproduction correction module. The inertial sensor module, the action database module, the human body action model establishment module, the action capture module, the data reading module and the action reproduction correction module are communicatively connected. The motion capture trajectory of the present invention has the smallest error compared with the Kinect method, which is significantly lower than the traditional method. Finally, through the action recognition comparison results, it can be seen that compared with the action recognition system based on Kinect, the recognition accuracy of this system generally maintains at 92% and above, and the maximum value of the recognition accuracy can reach 97.6%. This proves that the performance of this system is more superior than that of the action recognition system based on Kinect.

Description

Technical Field

[0001] The present invention relates to the technical field of motion capture, and in particular to a white-edge coordinate system for a camer module. Background Art

[0002] Motion capture is to set trackers at key parts of a moving object. The technology involves data such as dimensional measurement, positioning and orientation determination of objects in physical space that can be directly understood and processed by a computer. Trackers are set at key parts of the moving object, and the Motioncapture system captures the positions of the trackers, and then three-dimensional space coordinate data is obtained after being processed by a computer. When the data is recognized by the computer, it can be applied in fields such as animation production, gait analysis, biomechanics, and ergonomics.

[0003] The original light gun game could obtain the data of our movement by capturing the line scan of a CRT TV. In recent years, due to the popularity of LCD TVs and projectors, it is impossible to capture coordinate data. So, an infrared SensorBar is placed in front of the TV to capture coordinate data through a Camera, or relative data of movement is obtained through a gyroscope, etc. through an attitude algorithm. However, these two methods can only obtain relative displacement data and cannot obtain absolute coordinates, and it is impossible to achieve getting the coordinates wherever you point, so the experience effect is very poor and it has not been recognized by a large number of customers in light guns. Therefore, it is necessary to design a white-edge coordinate system for a camer module to solve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies in the prior art and propose a white-edge coordinate system for a camer module.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A white-edge coordinate system for a camer module includes an inertial sensor module, an action database module, a human action model establishment module, an action capture module, a data reading module and an action reproduction and correction module. The inertial sensor module, the action database module, the human action model establishment module, the action capture module, the data reading module and the action reproduction and correction module are communicatively connected;

[0007] The inertial sensor module captures and fuses human action information to calculate human posture information;

[0008] The action database module is responsible for storing and reproducing actions;

[0009] The human action model establishment module is responsible for processing action information and feature extraction for accurate action recognition.

[0010] As a preferred technical solution of the present invention, the inertial sensor module includes four parts: a magnetometer, a gyroscope, an accelerometer, and a positioning technology. Before performing motion capture and recognition, it is first necessary to construct a human motion database and a three-dimensional human joint model, use this model to reconstruct the motion information of the dancer, capture the motion through the inertial sensor, and finally input the captured motion into the three-dimensional model, compare it with the database motion, and finally obtain the accurate motion to achieve accurate motion recognition.

[0011] As a preferred technical solution of the present invention, after the motion database module obtains the motion information in the database, it matches the standard motion result and adjusts and corrects the motion according to this motion. If the human motion database is represented as DT, where DT = X1, X2,..., X N , X and N respectively represent the motion and the total number of video frames in the motion. Thus, the N-frame motion data can be expressed as: X = F1, F2,..., F N , F1 and F N respectively represent the first-frame data and the Nth-frame data, and T represents the filtering compensation value.

[0012] As a preferred technical solution of the present invention, based on the constructed human motion database, the human motion model establishment module constructs a two-dimensional model and a three-dimensional model of human joint motions respectively to facilitate subsequent motion reproduction and recognition. The human motion joints mainly include 21 in total, such as the head, neck, left and right shoulders, left and right elbows, etc., and the number of bones is 20. By combining the two data, a two-dimensional model of human motion is obtained. To accurately match the human motion in the database, the length of the human bones is obtained from the joint point model, and the human motion model is adjusted according to the specific length of the bones to obtain the accurate human proportion and recognition accuracy. Let the height of the two-dimensional model of human motion be I, then the ratios of the forearm to the upper arm can be calculated as 0.2111I and 0.1242I respectively, the length-width ratios of the head and neck to the shoulders are 0.1763I and 0.1322I respectively, the ratios of the torso to the hip width are 0.2251I and 0.1581I respectively, and the ratios of the thigh to the calf are 0.3031I and 0.2533I respectively. After obtaining the specific ratio information of the lengths of each bone, the bone positions are corresponded to the motions one by one and converted into the form of three-dimensional animation.

[0013] As a preferred technical solution of the present invention, the inertial sensor module selects an ADXL345 model acceleration sensor to capture actions. This sensor has the advantages of strong stability and high integration, and is very suitable for dynamic acceleration with a large impact. The gyroscope selects an L3G4200D model sensor with higher sensitivity, which can also be called an angular velocity sensor. When the measurement range of this sensor reaches ±250 dps, its sensitivity can reach 8.75 mdps / digit. Therefore, the high-sensitivity characteristic of this sensor makes it very suitable for action capture. An HMC5883L magnetoresistive sensor is added to the action sensor to detect magnetic field errors and improve the capture accuracy and deviation correction ability of the action sensor.

[0014] As a preferred technical solution of the present invention, the data reading module mainly includes 6 steps, specifically:

[0015] (1) Before data reading, the inertial sensor needs to be connected to the computer using USB, and the serial port of the sensor is opened so that its baud rate reaches 115;

[0016] (2) The sensor data is read using a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer respectively;

[0017] (3) Cyclic redundancy check is performed through CRC16;

[0018] (4) The verified data is fused using a data fusion algorithm to obtain the initial action data;

[0019] (5) The initial data is placed into the queue Quenue, and the action posture is further read in this queue;

[0020] (6) And the constructed three-dimensional human action model is driven using this posture data, and finally the posture data thread reading is completed.

[0021] As a preferred technical solution of the present invention, the action reproduction correction module can input the action capture data of the above action sensor into the constructed two-dimensional human action model, and use the distance metric method to compare and match the standard actions in the database, so as to find out the incorrect actions of the dance trainer and correct and adjust them according to the standard actions. When matching the joint points of the captured action and the database action, since the acting forces of each joint point are different, the nodes of the reproduction model are also different. Therefore, in order to increase the matching degree of each joint point and reduce the model reproduction error, the hip joint of the dance trainer is selected as the reference point in the distance metric to determine the key positions of each joint point, and the corresponding threshold can be set. The action position error is judged by this threshold. If the action is greater than the standard threshold, it means that the action has a large deviation, and the system can issue a prompt to display the position of the action deviation, thereby completing the automatic and accurate recognition and correction of the action.

[0022] The action automatic recognition system based on the action capture sensor designed by the present invention is feasible and effective. Through this system, different forms of actions can be accurately captured and recognized. The action capture joint points and angles of the action sensor adopted by this system are closer to the Kinect standard values, indicating that the action sensor can achieve accurate capture and positioning of actions. At the same time, by comparing the motion trajectories of different methods, it is found that the motion capture trajectory of this method has the smallest error with the Kinect method, which is significantly lower than the traditional method. Finally, through the action recognition comparison results, it can be seen that compared with the action recognition system based on Kinect, the recognition accuracy of this system generally maintains at 92% and above, and the maximum recognition accuracy can reach 97.6%. This proves that the performance of this system is more superior than that of the action recognition system based on Kinect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic structural diagram of a white edge coordinate system of a camer module proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0025] Refer to Figure 1 , a white edge coordinate system of a camer module, including an inertial sensor module, an action database module, a human action model establishment module, an action capture module, a data reading module and an action reproduction correction module. The inertial sensor module, the action database module, the human action model establishment module, the action capture module, the data reading module and the action reproduction correction module are communicatively connected;

[0026] The inertial sensor module captures and fuses human motion information to calculate human posture information;

[0027] The motion database module is responsible for storing and reproducing motions;

[0028] The human motion model establishment module is responsible for processing motion information and feature extraction for accurate motion recognition.

[0029] Refer to Figure 1 , the inertial sensor module includes four parts: a magnetometer, a gyroscope, an accelerometer, and a positioning technology. Before motion capture and recognition, it is necessary to first construct a human motion database and a three-dimensional human joint model, use this model to reconstruct the motion information of the dancer, capture the motion through the inertial sensor, and finally input the captured motion into the three-dimensional model, compare it with the database motion, and finally obtain the accurate motion to achieve accurate motion recognition.

[0030] Refer to Figure 1 , after the motion database module obtains the motion information in the database, it matches the standard motion result, and adjusts and corrects the motion according to this motion. If the human motion database is represented as DT, where DT = X1, X2,..., X N , X and N respectively represent the motion and the total number of video frames in the motion. Thus, the N-frame motion data can be expressed as: X = F1, F2,..., F N , F1 and F N respectively represent the first-frame data and the Nth-frame data, and T represents the filtering compensation value.

[0031] Refer to Figure 1 , based on the constructed human motion database, the human motion model establishment module will respectively construct a two-dimensional model and a three-dimensional model of human joint motions for subsequent motion reproduction and recognition. The human motion joints mainly include 21 in total, such as the head, neck, left and right shoulders, left and right elbows, etc., and the number of bones is 20. By combining the two data, a two-dimensional model of human motion is obtained. To accurately match the human motions in the database, the human bone lengths will be obtained from the joint point model, and the human motion model will be adjusted according to the specific lengths of these bones to obtain accurate human proportions and recognition accuracy. Let the height of the two-dimensional model of human motion be I, then the ratios of the forearm to the upper arm can be calculated as 0.2111I and 0.1242I respectively, the length-width ratios of the head and neck to the shoulders are 0.1763I and 0.1322I respectively, the ratios of the torso to the hip width are 0.2251I and 0.1581I respectively, and the ratios of the thigh to the calf are 0.3031I and 0.2533I respectively. After obtaining the specific ratio information of each bone length, the bone positions will be corresponding to the motions one by one and converted into the form of three-dimensional animation.

[0032] Reference Figure 1 Figure 1 , the inertial sensor module selects the ADXL345 model acceleration sensor to capture actions. This sensor has the advantages of strong stability and high integration, and is very suitable for dynamic acceleration with a large impact. The gyroscope selects the L3G4200D model sensor with higher sensitivity, which can also be called an angular velocity sensor. When the measuring range of this sensor reaches ±250 dps, its sensitivity can reach 8.75 mdps / digit. Therefore, the high-sensitivity feature of this sensor makes it very suitable for action capture. The HMC5883L magnetoresistive sensor is added to the action sensor to detect magnetic field errors and improve the capture accuracy and deviation correction ability of the action sensor.

[0033] Reference Figure 1 Figure 1 , the data reading module is mainly divided into 6 steps, specifically expressed as:

[0034] (1) Before reading data, it is necessary to connect the inertial sensor to the computer using USB and open the serial port of the sensor so that its baud rate reaches 115;

[0035] (2) Read the sensor data using a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer respectively;

[0036] (3) Perform cyclic redundancy check through CRC16;

[0037] (4) Use the data fusion algorithm to fuse the verified data to obtain the initial action data;

[0038] (5) Place the initial data into the queue Quenue and further read the action posture in this queue;

[0039] (6) And use the posture data to drive the constructed three-dimensional human action model to finally complete the reading of the posture data thread.

[0040] Reference Figure 1, based on the motion capture data of the above motion sensors, the motion reproduction correction module can input this data into the constructed two-dimensional human motion model, and use the distance metric method to compare and match the standard motions in the database, so as to find out the incorrect motions of the dance trainer and correct and adjust them according to the standard motions. When matching the joint points of the captured motion and the database motion, since the acting forces of each joint point are different, the nodes of the reproduction model are also different. Therefore, to increase the matching degree of each joint point and reduce the model reproduction error, the hip joint of the dance trainer is selected as the reference point in the distance metric, and the key positions of each joint point are determined, and then the corresponding thresholds can be set. The motion position error is judged through this threshold. If the motion is greater than the standard threshold, it means that the motion has a large deviation, and the system can issue a prompt to display the position of the motion deviation, thus completing the automatic and accurate recognition and correction of the motion.

[0041] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A white edge coordinate system for a camer module, characterized in that, It includes an inertial sensor module, an action database module, a human action model establishment module, an action capture module, a data reading module, and an action reproduction and correction module. The inertial sensor module, the action database module, the human action model establishment module, the action capture module, the data reading module, and the action reproduction and correction module are communicatively connected; The inertial sensor module captures and fuses human action information to calculate human posture information; The action database module is responsible for storing and reproducing actions; The human action model establishment module is responsible for processing action information and feature extraction for accurate action recognition; Based on the constructed human action database, the human action model establishment module constructs a two-dimensional model and a three-dimensional model of human joint actions respectively for subsequent action reproduction and recognition. The main human action joints include the head, neck, left and right shoulders, left and right elbows, etc., a total of 21, and the number of bones is 20. By combining the two data, a two-dimensional model of human actions is obtained. To accurately match the human actions in the database, the human bone lengths are obtained from the joint point model, and the human action model is adjusted according to the specific lengths of the bones to obtain accurate human proportions and recognition accuracy. Let the height of the two-dimensional human action model be I, then the ratios of the forearm to the upper arm can be calculated as 0.2111I and 0.1242I respectively, the length-width ratios of the head and neck to the shoulders are 0.1763I and 0.1322I respectively, the ratios of the torso to the hip width are 0.2251I and 0.1581I respectively, and the ratios of the thigh to the calf are 0.3031I and 0.2533I respectively. After obtaining the specific ratio information of each bone length, the bone positions are corresponded to the actions one by one and converted into three-dimensional animation form; Therefore, to increase the matching degree of each joint point and reduce the model reproduction error, the hip joint of the dance trainer is selected as the reference point in the distance metric to determine the key positions of each joint point, and then the corresponding thresholds can be set. The action position error is judged through this threshold. If the action is greater than the standard threshold, it means that the action has a large deviation, and the system can issue a prompt to display the action deviation position, thus completing the automatic and accurate recognition and correction of the action.

2. The white edge coordinate system for a camer module according to claim 1, characterized in that, The inertial sensor module includes four parts: a magnetometer, a gyroscope, an accelerometer, and a positioning technology. Before performing action capture and recognition, first, a human action database and a three-dimensional human joint model need to be constructed. The action information of the dancer is reconstructed using this model, and the action is captured by the inertial sensor. Finally, the captured action is input into the three-dimensional model and compared with the database actions to obtain accurate actions for accurate action recognition.

3. The white edge coordinate system for a camer module according to claim 1, characterized in that, After the action database module obtains the action information in the database, it matches the standard action result, and performs action adjustment and correction according to this action. If the human body action database is represented as DT, where DT = X1, X2,..., X N , X and N respectively represent the action and the total number of video frames in the action. Thus, the N-frame action data can be expressed as: X = F1, F2,..., F N , F1 and F N respectively represent the first-frame data and the Nth-frame data, and T represents the filtering compensation value.

4. The white edge coordinate system for a camer module according to claim 1, characterized in that, The inertial sensor module selects the ADXL345 acceleration sensor to capture actions. This sensor has the advantages of strong stability and high integration, and is very suitable for dynamic acceleration with large impacts. The gyroscope selects the L3G4200D sensor with high sensitivity, which can also be called an angular velocity sensor. When the range of this sensor reaches ±250 dps, its sensitivity can reach 8.75 mdps / digit. Therefore, the high-sensitivity feature of this sensor makes it very suitable for action capture. The HMC5883L magnetoresistive sensor is added to the action sensor to detect magnetic field errors and improve the capture accuracy and deviation correction ability of the action sensor.

5. The white edge coordinate system for a camer module according to claim 1, characterized in that, The data reading module is mainly divided into 6 steps, specifically expressed as: (1) Before reading data, the inertial sensor needs to be connected to the computer using USB, and the serial port of the sensor needs to be opened so that its baud rate reaches 115; (2) The triaxial accelerometer, triaxial gyroscope, and triaxial magnetometer are respectively used to read the sensor data; (3) Cyclic redundancy check is performed through CRC16; (4) The data fusion algorithm is used to fuse the verified data to obtain the initial action data; (5) The initial data is placed in the queue Quenue, and the action posture is further read in this queue; (6) And the constructed three-dimensional human action model is driven by the posture data, and finally the posture data thread reading is completed.

6. The white edge coordinate system for a camer module according to claim 1, characterized in that, Based on the action capture data of the above action sensor, the action reproduction and correction module can input this data into the constructed two-dimensional human action model, and use the distance metric method to compare and match the standard actions in the database, so as to find out the wrong actions of the dance trainer and correct and adjust according to the standard actions. When matching the joint points of the captured action and the database action, since the acting forces of each joint point are different, the nodes of the reproduction model are also different.

Images