Data glove calibration method and system, electronic device and computer storage medium

Abstract

The application provides a data glove calibration method and system, an electronic device and a computer storage medium, comprising: a target hand and a reference hand synchronously performing corresponding hand actions; reference posture information of the reference hand is collected by using a visual processing device; sensing posture information of the target hand synchronously collected by a data glove is obtained; the reference posture information and the sensing posture information are time-aligned; and calibration data of the data glove is established by using the time-aligned reference posture information and sensing posture information. The reference posture information is obtained by detecting visual key points of the hand of the same user without wearing the data glove, which can avoid the problem of hand key point extraction error caused by the shielding of the glove and the like, ensure data reliability, and improve calibration accuracy; the calibration data is established by time alignment, which can avoid time misalignment caused by different sampling rates, ensure data consistency, further improve calibration accuracy, and solve the problem of poor accuracy of the existing data glove calibration scheme.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data glove calibration method and system, electronic device, and computer storage medium. Background Technology

[0002] Data gloves typically use bend sensors, inertial sensors, pressure sensors, magnetic sensors, or other flexible sensors to detect finger flexion, finger spacing, or hand posture. Because different users have different hand sizes, finger lengths, wearing tightness, and sensor placement, the actual joint angle corresponding to the same sensor output value may differ. Therefore, user-level calibration is usually required before using data gloves.

[0003] However, existing data glove calibration schemes suffer from low data adaptability due to variations in stretching, bending, and misalignment of flexible sensors caused by different hand shapes and hand movements, resulting in unsatisfactory calibration results. While vision-based methods can acquire key glove points and improve calibration accuracy to some extent, existing vision-based calibration schemes are susceptible to errors in key point extraction due to factors such as glove color, material, shape, and occlusion, leading to poor calibration accuracy. Furthermore, different gloves require re-collecting samples to train the vision system, resulting in high calibration costs and low adaptability. In addition, visual data is often difficult to precisely correlate with sensor data in time, and the data communication methods of different gloves and vision systems also differ, leading to larger calibration errors and poor scalability. Summary of the Invention

[0004] The purpose of this invention is to provide a data glove calibration method and system, electronic device and computer storage medium, so as to at least solve the problem of poor accuracy of existing data glove calibration schemes.

[0005] To address the aforementioned technical problems, this invention provides a data glove calibration method, comprising: The target hand and the reference hand perform corresponding hand movements synchronously, wherein the target hand is the user's hand wearing a data glove, and the reference hand is the user's hand not wearing a data glove; The reference posture information of the reference hand is collected using a vision processing device; Acquire the target hand's posture information synchronously collected by the sensors on the data glove; Timing alignment of reference pose information and sensor pose information; Calibration data for the data glove is established using the time-aligned reference posture information and sensor posture information.

[0006] Optionally, in the data glove calibration method, the hand movements include finger-to-finger, finger-to-finger, and fist-clenching movements.

[0007] Optionally, in the data glove calibration method, the reference posture information includes key points of the hand and finger joint bending angles and the separation between adjacent fingers calculated based on the key points of the hand.

[0008] Optionally, in the data glove calibration method, the key points of the hand include the wrist key point, thumb joint key point, index finger joint key point, middle finger joint key point, ring finger joint key point, and little finger joint key point.

[0009] Optionally, in the data glove calibration method, the method for calculating the finger joint flexion angle and the separation between adjacent fingers based on the key points of the hand includes: The finger joint bending angle is calculated based on the spatial angle between adjacent key points on the finger. The separation between adjacent fingers is calculated based on the distance between key points on the hand at the tips of adjacent fingers.

[0010] Optionally, in the data glove calibration method, the method for timing alignment of the reference posture information and the sensing posture information includes: Obtain the first timestamp of the target hand movement in the reference posture information; Obtain the second timestamp of the target hand movement in the sensor posture information; Determine the time offset using the first and second timestamps; The timestamp of the sensing pose information is corrected based on the time offset to align the reference pose information and the sensing pose information in time.

[0011] Optionally, in the data glove calibration method, the method for timing alignment of the reference posture information and the sensing posture information further includes: If a sensor frame exists within a preset time period of the first timestamp of a video frame, the sensor frame with the smallest time difference from the first timestamp will be matched with the video frame, and the second timestamp of the matched sensor frame will be obtained. If there are no sensor frames within the preset time period of the first timestamp of the video frame, then the timestamps and sensing posture information of the sensor frames adjacent to the video frame before and after it are linearly interpolated to obtain the second timestamp and sensing posture information of the video frame.

[0012] Optionally, in the data glove calibration method, the calibration data includes a fitting function between the sensor values ​​of the data glove and the bending angle of the finger joints, a fitting function between the sensor values ​​of the data glove and the separation between adjacent fingers, and / or a calibration model.

[0013] To address the aforementioned technical problems, the present invention also provides a data glove calibration system for implementing the data glove calibration method as described in any of the preceding claims, wherein the data glove calibration system comprises: The information acquisition module is used to acquire reference posture information of the reference hand using a vision processing device, and to acquire the sensor posture information of the target hand synchronously acquired by the sensors on the data glove. The target hand and the reference hand synchronously perform corresponding hand movements. The target hand is the user's hand wearing the data glove, and the reference hand is the user's hand not wearing the data glove. The timing synchronization module is used to perform timing alignment between reference posture information and sensor posture information; The data calibration module is used to establish calibration data for the data glove using time-aligned reference posture information and sensor posture information.

[0014] To address the aforementioned technical problems, the present invention also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and executable by the processor; when the processor runs the executable program, it performs the data glove calibration method as described in any of the preceding claims.

[0015] To address the aforementioned technical problems, the present invention also provides a computer storage medium storing an executable program; when the executable program is executed, it implements the data glove calibration method as described in any of the preceding claims.

[0016] The present invention provides a data glove calibration method and system, electronic device, and computer storage medium, comprising: synchronously performing corresponding hand movements on a target hand and a reference hand, wherein the target hand is the user's hand wearing a data glove, and the reference hand is the user's hand not wearing a data glove; acquiring reference posture information of the reference hand using a vision processing device; acquiring sensor posture information of the target hand synchronously acquired by sensors on the data glove; performing time-series alignment of the reference posture information and the sensor posture information; and establishing calibration data for the data glove using the time-series aligned reference posture information and sensor posture information. By obtaining reference posture information through visual keypoint detection on the same user's hand not wearing a data glove, the problem of incorrect keypoint extraction due to glove occlusion can be avoided, ensuring the reliability of the reference posture information and improving calibration accuracy. By performing time-series alignment of the reference posture information and the sensor posture information and establishing calibration data, time misalignment caused by different sampling rates can be avoided, ensuring the consistency of information data and further improving calibration accuracy, thus solving the problem of poor accuracy in existing data glove calibration schemes. Attached Figure Description

[0017] Figure 1 A flowchart of the data glove calibration method provided in this embodiment; Figure 2 This is a schematic diagram of key hand points provided in this embodiment; Figure 3 This is a schematic diagram illustrating the calculation principle of the finger joint bending angle provided in this embodiment; Figure 4 This is a schematic diagram illustrating the calculation principle of the separation amount between adjacent fingers provided in this embodiment; Figure 5 This is a schematic diagram illustrating the timing alignment of the visual frame and the sensor frame provided in this embodiment; Figure 6 This is a structural block diagram of the data glove calibration system provided in this embodiment. Detailed Implementation

[0018] The data glove calibration method and system, electronic device, and computer storage medium proposed in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and use non-precise scales, used only to facilitate and clarify the illustration of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structure. In particular, different figures may emphasize different aspects and sometimes use different scales.

[0019] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this invention are used to distinguish similar objects in order to describe embodiments of the invention, and are not used to describe a specific order or sequence. It should be understood that such uses of terminology are interchangeable where appropriate. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Current calibration methods for data gloves typically require users to perform several fixed postures, such as with the palm open, the fist clenched, or the fingers spread, and then perform linear interpolation or fixed parameter mapping based on the sensor values ​​in these postures. However, the actual sensor output and finger joint angles are often not linearly related, especially when flexible sensors are affected by stretching, bending, or changes in wearing position. Simple two- or three-point calibration is insufficient to cover the entire movement process. Furthermore, traditional calibration methods usually only consider the finger bending angle and rarely consider the distance between the fingers.

[0021] While cameras and hand keypoint detection models can be used to obtain the positions of hand keypoints and further estimate finger joint angles and finger spacing, the appearance of the hand can be altered by the color, material, occlusion, reflection, or structural components of the glove after it is worn. This makes it difficult for general hand keypoint detection models to reliably identify the skeleton of the gloved hand. Forcing the identification of the gloved hand can easily lead to keypoint loss, misidentification of fingers, or abrupt changes in joint angles, thus affecting the quality of calibration data.

[0022] In addition, some vision-based recognition solutions attempt to directly identify gloved hands or collect training datasets for specific glove appearances. However, different data gloves may have different colors, materials, thicknesses, sensor arrangements, and shapes. Once the glove model or appearance is changed, the original recognition model may need to collect samples and be retrained, which is costly and difficult to adapt to data gloves from different manufacturers or with different protocols as a general calibration tool.

[0023] Furthermore, in practical applications, the video capture frame rate and the data glove sensor sampling rate are often different. For example, video may be around 30 frames per second, while serial port sensor data may be around 10 to 20 frames per second, making it difficult to accurately match visual data and sensor data in time. If matching is done directly based on the most recent frame, it may cause misalignment of angles and sensor values, especially when the user moves quickly, the error will be more obvious. Different data gloves may also have different data formats, number of sensors, and communication interfaces. If the calibration system hardcodes the parsing rules and the number of sensors, the scalability will be poor.

[0024] Based on the aforementioned shortcomings of existing data glove calibration schemes, this embodiment provides a data glove calibration method, such as... Figure 1 As shown, it includes: S1, the target hand and the reference hand synchronously perform corresponding hand movements, wherein the target hand is the user's hand wearing a data glove, and the reference hand is the user's hand not wearing a data glove; S2, using a vision processing device to collect reference hand posture information; S3, acquire the target hand's sensor posture information synchronously collected by the sensors on the data glove; S4, perform timing alignment of reference posture information and sensor posture information; S5 uses the time-aligned reference posture information and sensor posture information to establish calibration data for the data glove.

[0025] The data glove calibration method provided in this embodiment obtains reference posture information by performing visual key point detection on the same user's hand without a data glove. This avoids errors in key point extraction caused by glove occlusion, ensuring the reliability of the reference posture information and improving calibration accuracy. By aligning the reference posture information and the sensor posture information in time and establishing calibration data, it avoids time misalignment caused by different sampling rates, ensuring the consistency of information data and further improving calibration accuracy. This solves the problem of poor accuracy in existing data glove calibration schemes.

[0026] Specifically, in this embodiment, in step S1, the target hand and the reference hand synchronously perform corresponding hand movements, wherein the target hand is the user's hand wearing a data glove, and the reference hand is the user's hand not wearing a data glove.

[0027] In this embodiment, the target hand and the reference hand synchronously perform the same or mirror-image hand movements. Since the shape and movements of the two hands of the same user are highly consistent, this embodiment uses the user's hand without data gloves as the reference hand. This not only ensures synchronized and consistent hand movements, guaranteeing a high degree of consistency in the intrinsic information of the two hands, but also, because the reference hand is not obstructed by data gloves, allows for precise location of key hand points using a visual processing device and the acquisition of accurate hand posture information, ensuring the accuracy of subsequent data glove calibration.

[0028] In this embodiment, the hand movements include finger-joining, finger-spreading, and fist-clenching. The finger-joining and finger-spreading movements can be combined, i.e., performing a finger-joining and spreading movement simultaneously. The finger-spreading movement can include a slow spreading movement, where the duration from the start to the end of the movement exceeds a first preset duration. The fist-clenching movement can include a rapid fist-clenching movement and a slow fist-clenching movement, where the duration of the rapid fist-clenching movement does not exceed a second preset duration, and the duration of the slow fist-clenching movement exceeds a third preset duration. The finger-spreading movement can include a fully extended finger-spreading movement, i.e., all fingers are opened to the user's maximum position.

[0029] In practical applications, the duration of each preset time can be customized based on user characteristics, thereby defining the speed. Furthermore, different hand gestures can be set according to actual needs; this application does not impose any restrictions on this.

[0030] Preferably, in order to accurately extract the reference posture information and sensor posture information corresponding to the hand movements, in this embodiment, a state machine can be set up to guide the user to complete the corresponding hand movements with both hands.

[0031] For example, in the state machine standby state, the system prompts the user to synchronize the target hand and the reference hand, opening their palms and closing their fingers. When the reference hand is recognized as having an open palm and closed fingers for several consecutive frames, the system clears the old data and begins recording, entering a waiting state for a rapid fist clench.

[0032] While waiting for the rapid fist-clenching state, the system prompts the user to simultaneously and rapidly clench both fists. When the reference hand is detected as clenched, and the time between entering this state and detecting the fist is less than a first time threshold, the system records the rapid fist-clenching synchronization event. Since the target hand and reference hand perform the rapid fist-clenching action synchronously, this event can be used to estimate the time offset between the reference hand's visual data and the target hand's sensor data, thereby accurately obtaining the time offset using short-duration actions.

[0033] While waiting for the slow fist-clenching state, the system prompts the user to simultaneously and slowly clench both hands from an open-fingers state. As the reference hand moves from an open-fingers state and gradually reaches a clenched fist state, the duration of the action falls between a second and a third time threshold. Because this action is slow, more continuous sampling points covering different bending angles can be obtained; therefore, the data from this stage can be used as non-linear calibration samples for finger bending angles.

[0034] While waiting for the hands to slowly open, the system prompts the user to gradually open both hands from a clenched fist position until the reference hand returns to an open-fingered state. Data from this stage can be used to acquire the return angle curve and also to determine if the sensor exhibits hysteresis.

[0035] While waiting for the fingers to fully open, the system prompts the user to gradually open both hands from a finger-to-finger position until the fingers are fully separated. The system uses the data between the finger-to-finger closing event and the finger-to-finger full opening event as a calibration sample for the finger separation amount, which is used to establish a mapping relationship between sensor values ​​and the separation amount of adjacent fingers or hand posture parameters.

[0036] If the user's hands are not detected to make corresponding movements within the specified time, a timeout will be indicated and the calibration will be considered a failure to prevent calibration errors caused by incorrect calibration data extraction.

[0037] Furthermore, in this embodiment, step S2 involves using a vision processing device to collect reference posture information of the reference hand.

[0038] Specifically, in this embodiment, an existing vision processing device with image acquisition capabilities can be used to acquire reference posture information of the examinee's hand. Typically, the vision processing device includes at least one camera and a processor. The camera is used to capture images of the reference hand; the processor has a built-in key point detection model for image analysis of the reference hand images captured by the camera to obtain reference posture information.

[0039] Furthermore, in practical applications, the palm of the reference hand can be positioned directly facing the camera to reduce occlusion and perspective errors. The target hand can be located within or outside the camera's field of view. Even if the target hand is within the field of view, the system will still use the reference hand's key points as the angle reference because data gloves may cause visual key point detection to fail.

[0040] In this embodiment, the reference posture information includes key hand points and finger joint flexion angles and the distance between adjacent fingers calculated based on these key hand points. The finger joint flexion angles and the distance between adjacent fingers serve as visual reference values ​​for the synchronized movements of the target hand. For example, Figure 2 As shown, the key points of the hand include 21 key points such as the wrist key point, thumb joint key point, index finger joint key point, middle finger joint key point, ring finger joint key point, and little finger joint key point.

[0041] Specifically, in this embodiment, the finger joint bending angle can be calculated based on the spatial angle between adjacent key points on the finger. For example, as Figure 2 and Figure 3 As shown, for a certain finger joint, let the keypoint of the joint to be calculated be B, the proximal keypoint be A, the distal keypoint be C, and the angle between vectors BA and BC be angle, then we have:

[0042] The finger joint bending angle bend is obtained as bend = 180° - angle.

[0043] Furthermore, in this embodiment, the separation between adjacent fingers is calculated based on the distance between key hand points at the fingertips of adjacent fingers. Specifically, to accommodate different hand shapes, the distance between the fingertips of adjacent fingers can be normalized according to the palm size. The palm size can be determined based on the distance between the wrist key point and the metacarpophalangeal joint key point of the middle finger; for the separation between the index and middle fingers, the Euclidean distance between the fingertips of the index and middle fingers can be divided by the palm size; the separation between the middle and ring fingers, and between the ring and little fingers, can be calculated in the same way. For example, as... Figure 2 and Figure 4 As shown, the palm scale can be taken as the Euclidean distance between the wrist keypoint P0 and the middle finger metacarpophalangeal joint keypoint P9, and the normalized separation between the index and middle fingers can be represented as gap. index-middle =d 8-12 / palm_scale, the normalized separation between the middle and ring fingers can be represented as gap. middle-ring =d 12-16 / palm_scale, the normalized separation between the ring and little fingers, can be represented as gap. ring-pinky =d 16-20 / palm_scale.

[0044] Preferably, the vision processing device can also classify the current hand movement based on the geometric relationships of key points of the reference hand. Specifically, an open palm with fingers together can be determined by the degree of finger extension and the distance between adjacent fingertips; an open palm with fingers fully separated can be determined by the distance between adjacent fingertips and the distance or angle between the thumb and index finger; and a clenched fist can be determined by the bending angles of multiple fingers and the distance from the fingertips to the center of the palm. This allows for further determination of whether the user's hand movement matches the movement required by the state machine, thereby ensuring the reliability of the data glove calibration.

[0045] More preferably, in this embodiment, a sensitivity parameter can also be calculated based on the reference posture information. The larger the sensitivity parameter, the greater the allowable distance between adjacent fingertips, the wider the allowable range of finger extension angles, and the easier it is to identify the reference hand as having fingers closed and spread. Conversely, the smaller the sensitivity parameter, the stricter the identification, requiring adjacent fingertips to be closer and fingers to be straighter for the reference hand to be identified as having fingers closed and spread. The sensitivity parameter can adapt to different user hand shapes, natural reference hand postures, camera angles, and lighting conditions, improving the adaptability of the data glove calibration method.

[0046] Furthermore, in this embodiment, step S3 involves acquiring the sensor posture information of the target hand synchronously collected by the sensors on the data glove.

[0047] Specifically, in this embodiment, since the data glove integrates various sensors, the data collected by the sensors can be transmitted via serial port, USB, Bluetooth, network, or other communication methods according to the sensor type to obtain the sensing posture information. The specific methods for obtaining the sensing posture information are known to those skilled in the art in the prior art, and will not be elaborated here.

[0048] Furthermore, in this embodiment, step S4 involves timing alignment of the reference posture information and the sensing posture information.

[0049] In this embodiment, the target hand movement can be a rapid fist-clenching motion.

[0050] Specifically, in this embodiment, firstly, the first timestamp of the target hand movement in the reference posture information is obtained, which can be directly obtained from the vision processing device; then, the second timestamp of the target hand movement in the sensing posture information is obtained, which can be obtained by detecting rapid change points of the sensors on the data gloves; then, the time offset is determined using the first and second timestamps; finally, the timestamp of the sensing posture information is corrected according to the time offset to perform time alignment between the reference posture information and the sensing posture information.

[0051] For example, such as Figure 5 As shown, a rapid fist-clenching visual event was captured at video frame V3, and a rapid change point was captured in sensor frame S2, i.e., the first timestamp is t(V3) and the second timestamp is t(S2). The time offset is set to offset the sensor frame as a whole according to the time offset, so that video frame V3 is aligned with sensor frame S2.

[0052] In practical applications, considering the differences between video frames and sensor frames, in order to further improve the accuracy of timing alignment and thus the accuracy of subsequent calibration results, in this embodiment, as follows: Figure 5 As shown, if a sensor frame exists within a preset time period of the first timestamp of a video frame, the sensor frame with the smallest time difference from the first timestamp is matched with the video frame, and the second timestamp of the matched sensor frame is obtained; if no sensor frame exists within the preset time period of the first timestamp of a video frame, the timestamps and sensing posture information corresponding to the adjacent sensor frames before and after the video frame are linearly interpolated to obtain the second timestamp and sensing posture information corresponding to the video frame.

[0053] Preferably, to ensure the validity of the data, in this embodiment, if the video frame is located before the beginning or after the end of the valid sensor data, it is retained as a null value or marked as unusable.

[0054] Furthermore, in this embodiment, step S5 involves using the time-aligned reference posture information and sensor posture information to establish calibration data for the data glove.

[0055] Specifically, in this embodiment, the calibration data includes a fitting function between the sensor values ​​of the data glove and the bending angle of the finger joints, a fitting function between the sensor values ​​of the data glove and the separation between adjacent fingers, and / or a calibration model.

[0056] In practical applications, the timestamp, frame number, state machine state, gesture classification, reference hand's knuckle bending angle and the separation between adjacent fingers, target hand's sensor value, time offset, and key event markers for each video frame and sensor frame are all recorded in the calibration data. Key events include the start recording time point, the synchronization point of rapid fist clenching, the start point of slow fist clenching, the end point of slow fist clenching, the point of slow opening to closing, the start point of finger separation, the point of full opening, and the calibration completion point.

[0057] In this embodiment, taking the quadratic equation set model as an example, the fitting function between the sensor value of the data glove and the bending angle of the finger joint and the fitting function between the sensor value of the data glove and the separation amount between adjacent fingers can be expressed as y=ax²+bx+c, where x is the sensor value of at least one sensor in the data glove or a combination of multiple sensor values, y is the bending angle of the finger joint or the separation amount between adjacent fingers, and a, b, and c are fitting parameters determined based on calibration samples.

[0058] In practical applications, for fitting finger flexion angles, data from the slow fist-clenching phase is preferred; for hysteresis or return analysis, data from the slow opening phase is preferred; and for fitting finger separation, data from the phase from finger close-up to full opening is preferred. By sampling and modeling separately according to state machine phases, errors caused by mixing different action types into the same model can be reduced.

[0059] Preferably, the calibration model generated from the calibration data includes a fitting model of the sensor values ​​of the data glove with the bending angle of the finger joints or the separation between adjacent fingers. The fitting model can be a linear model, a polynomial model, a spline model, a lookup table interpolation model, a neural network model, or other regression models.

[0060] In practical applications, calibration data can be output in CSV file format.

[0061] The data glove calibration method provided in this embodiment uses a reference hand without gloves for visual key point detection, avoiding the problem of unstable recognition of the gloved hand skeleton due to data glove occlusion. This method uses synchronized hand movements, taking the visual angle of the reference hand as the angular reference of the gloved target hand, thus obtaining reliable calibration true values. Because visual recognition operates on the reference hand without data gloves, the color, material, shape, and sensor arrangement of the data glove are not included in the visual model input; therefore, it is not necessary to re-collect and retrain the visual recognition dataset for different gloves. This method establishes a synchronization anchor between visual data and sensor data through a rapid fist-clenching event, effectively reducing time misalignment caused by different sampling rates. By collecting continuous samples during slow fist-clenching and slow fist-opening processes, this method can better describe the nonlinear relationship between sensor output and joint angles. By separately collecting the movement segment from fingers together to fully open, this method can simultaneously calibrate the finger bending angle and finger separation. The data glove calibration method provided in this embodiment reduces invalid data caused by user errors by constraining the action sequence through a state machine. Furthermore, the data glove calibration method provided in this embodiment can be adapted to different data gloves by configuring the number of sensors, communication interface, and data packet parsing rules.

[0062] This embodiment also provides a data glove calibration system for implementing the data glove calibration method described above, such as... Figure 6 As shown, the data glove calibration system includes: The information acquisition module is used to acquire reference posture information of the reference hand using a vision processing device, and to acquire the sensor posture information of the target hand synchronously acquired by the sensors on the data glove. The target hand and the reference hand synchronously perform corresponding hand movements. The target hand is the user's hand wearing the data glove, and the reference hand is the user's hand not wearing the data glove. The timing synchronization module is used to perform timing alignment between reference posture information and sensor posture information; The data calibration module is used to establish calibration data for the data glove using time-aligned reference posture information and sensor posture information.

[0063] Specifically, in this embodiment, the information acquisition module includes a sensor acquisition unit and an image acquisition unit; the sensor acquisition unit is used to acquire the sensor posture information of the target hand synchronously acquired by the sensors on the data glove; the image acquisition unit includes a camera and a processor, the camera is used to capture a reference hand image; the processor has a built-in key point detection model, which is used to perform image analysis on the reference hand image captured by the camera to obtain reference posture information, including key points of the hand and the finger joint bending angle and the separation between adjacent fingers calculated based on the key points of the hand.

[0064] Furthermore, in this embodiment, the information acquisition module also includes a state machine; the state machine is used to guide the user's hands to complete the corresponding hand movements, so as to ensure that the hand movements performed by the reference hand and the target hand meet the expected requirements and are synchronous and consistent.

[0065] Furthermore, the timing synchronization module includes a timing alignment unit and an interpolation processing unit; the timing alignment unit is used to perform timing alignment on the reference pose information and the sensing pose information; the interpolation processing unit is used to perform linear interpolation on the timestamps and sensing pose information corresponding to the adjacent sensor frames before and after the video frame when there are no sensor frames within a preset time period of the first timestamp of the video frame, so as to obtain the second timestamp and sensing pose information corresponding to the video frame.

[0066] Additionally, the calibration data generated by the data calibration module includes a fitting function between the sensor values ​​of the data glove and the bending angle of the finger joints, a fitting function between the sensor values ​​of the data glove and the separation between adjacent fingers, and / or a calibration model.

[0067] The data glove calibration system provided in this embodiment obtains reference posture information by performing visual key point detection on the same user's hand without a data glove through an information acquisition module. This avoids errors in key point extraction caused by glove obstruction, ensuring the reliability of the reference posture information and improving calibration accuracy. Furthermore, the system uses a timing synchronization module to align the reference posture information and the sensor posture information, and establishes calibration data through a data calibration module. This avoids time misalignment caused by different sampling rates, ensuring data consistency and further improving calibration accuracy. This solves the problem of poor accuracy in existing data glove calibration schemes.

[0068] Furthermore, this embodiment also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the data glove calibration method as described above.

[0069] Furthermore, this embodiment also provides a computer storage medium storing an executable program; when the executable program is executed, it implements the data glove calibration method described above.

[0070] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. In addition, the different parts between embodiments can also be combined with each other, and this invention does not limit this.

[0071] This embodiment provides a data glove calibration method and system, electronic device, and computer storage medium, including: synchronously executing corresponding hand movements with a target hand and a reference hand, wherein the target hand is the user's hand wearing a data glove, and the reference hand is the user's hand not wearing a data glove; acquiring reference posture information of the reference hand using a vision processing device; acquiring sensor posture information of the target hand synchronously acquired by sensors on the data glove; performing time-series alignment of the reference posture information and the sensor posture information; and establishing calibration data for the data glove using the time-series aligned reference posture information and sensor posture information. By obtaining reference posture information through visual keypoint detection on the same user's hand not wearing a data glove, the problem of incorrect keypoint extraction due to glove occlusion can be avoided, ensuring the reliability of the reference posture information and improving calibration accuracy. By performing time-series alignment of the reference posture information and the sensor posture information and establishing calibration data, time misalignment caused by different sampling rates can be avoided, ensuring the consistency of information data and further improving calibration accuracy, thus solving the problem of poor accuracy in existing data glove calibration schemes.

[0072] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A data glove calibration method, characterized in that, include: The target hand and the reference hand perform corresponding hand movements synchronously, wherein the target hand is the user's hand wearing a data glove, and the reference hand is the user's hand not wearing a data glove; The reference hand's posture information is collected using a vision processing device; Acquire the target hand's posture information synchronously collected by the sensors on the data glove; Timing alignment of reference pose information and sensor pose information; Calibration data for the data glove is established using the time-aligned reference posture information and sensor posture information.

2. The data glove calibration method according to claim 1, characterized in that, The hand movements include closing the fingers, opening the fingers, and making a fist.

3. The data glove calibration method according to claim 1, characterized in that, The reference posture information includes key hand points and finger joint bending angles and the separation between adjacent fingers calculated based on the key hand points.

4. The data glove calibration method according to claim 3, characterized in that, The method for calculating the finger joint bending angle and the separation between adjacent fingers based on the key points of the hand includes: The finger joint bending angle is calculated based on the spatial angle between adjacent key points on the finger. The separation between adjacent fingers is calculated based on the distance between key points on the hand at the tips of adjacent fingers.

5. The data glove calibration method according to claim 1, characterized in that, The method for temporal alignment of reference pose information and sensor pose information includes: Obtain the first timestamp of the target hand movement in the reference posture information; Obtain the second timestamp of the target hand movement in the sensor posture information; Determine the time offset using the first and second timestamps; The timestamp of the sensing pose information is corrected based on the time offset to align the reference pose information and the sensing pose information in time.

6. The data glove calibration method according to claim 5, characterized in that, The method for temporal alignment of reference pose information and sensing pose information further includes: If a sensor frame exists within a preset time period of the first timestamp of a video frame, the sensor frame with the smallest time difference from the first timestamp is matched with the video frame, and the second timestamp of the matched sensor frame is obtained. If there are no sensor frames within the preset time period of the first timestamp of the video frame, then the timestamps and sensing posture information of the sensor frames adjacent to the video frame are linearly interpolated to obtain the second timestamp and sensing posture information of the video frame.

7. The data glove calibration method according to claim 1, characterized in that, The calibration data includes a fitting function between the sensor values ​​of the data glove and the bending angle of the finger joints, a fitting function between the sensor values ​​of the data glove and the separation between adjacent fingers, and / or a calibration model.

8. A data glove calibration system for implementing the data glove calibration method as described in any one of claims 1 to 7, characterized in that, The data glove calibration system includes: The information acquisition module is used to acquire reference posture information of the reference hand using a vision processing device, and to acquire the sensor posture information of the target hand synchronously acquired by the sensors on the data glove. The target hand and the reference hand synchronously perform corresponding hand movements. The target hand is the user's hand wearing the data glove, and the reference hand is the user's hand not wearing the data glove. The timing synchronization module is used to perform timing alignment between reference posture information and sensor posture information; The data calibration module is used to establish calibration data for the data glove using time-aligned reference posture information and sensor posture information.

9. An electronic device, characterized in that, It includes a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the data glove calibration method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores an executable program; when the executable program is executed, it implements the data glove calibration method as described in any one of claims 1 to 7.

Images