Device for collecting multi-mode body data
By installing a bracket and sampling device on the helmet, the problem of insufficient data acquisition accuracy in motion capture systems was solved, enabling efficient acquisition of multimodal data and supporting high-quality training of embodied intelligent models.
Patent Information
- Application Number
- CN202422160200.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2025-12-09
- Estimated Expiration
- 2034-09-04
AI Technical Summary
Existing motion capture systems suffer from insufficient accuracy in collecting embodied intelligence data, which affects the effectiveness of subsequent model training.
Design a helmet device that acquires multimodal data by mounting a first support, a second support, and a counterweight assembly on the helmet, and equipping it with sampling devices such as a RealSense RGBD camera, an RGB camera, an ultra-wide-angle camera, an IMU pose sensor, or a microphone array.
It enables the collection of multi-angle and multi-modal data, improves the accuracy and completeness of the data, and supports high-quality training of embodied intelligent models.
Smart Images
Figure CN223650978U_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The utility model relates to the field of body data acquisition technology, in particular to a device for collecting multimodal body data. BACKGROUND
[0002] Body intelligence is one of the hottest application directions in the AI field in recent years, which combines the research results of robots and large language model directions, and the core goal is to obtain a body intelligent entity with real-time interaction ability, autonomous motion ability and task planning ability through data learning. At the current stage, one of the most critical elements of body intelligence is high-quality, large-capacity and multi-scene body data, which is used for model training, model fine-tuning, model migration and other key tasks of body intelligence.
[0003] The existing body intelligence data mainly has three types: the first type is entity robot operation data, which is convenient for direct use in model learning and model deployment, and is usually collected by a camera and a robot built-in sensor; the second type is simulation robot data, which is usually deployed in a software simulation system and collected by a computer software program; the third type is motion capture data, which is usually collected by a third-view motion capture system or a first-person motion capture system composed of a small number of sensors.
[0004] The accuracy of data acquisition directly affects the results of subsequent model training for the motion capture system; therefore, how to design a device for conveniently collecting motion capture data becomes a technical problem to be solved. UTILITY MODEL CONTENT
[0005] The utility model aims at the problems in the background art and provides a device for collecting multimodal body data, which improves the existing helmet, increases a first support and a second support, and installs different sampling devices to simultaneously collect different modal information, thereby facilitating the collection of motion capture data.
[0006] The technical scheme of the utility model provides a device for collecting multimodal body data, which includes a helmet main body, a first support, a second support, a sliding assembly and a counterweight assembly.
[0007] The first support is arranged in a group and located at the front of the helmet main body; a RealSense RGBD camera is installed on the first support to capture image information of the front of the helmet main body.
[0008] The sliding assembly is symmetrically arranged on the left and right sides of the helmet main body; the second support is installed on the sliding assembly and relatively slides along the sliding assembly.
[0009] The counterweight assembly is fixed to the rear end of the helmet main body to level the center of gravity of the entire helmet main body to the geometric center of the helmet main body.
[0010] The helmet body is equipped with matching straps for securing it.
[0011] In an optional embodiment, the counterweight assembly integrates a power supply module for powering the sampling equipment; the power supply module is electrically connected to each electrical device via wires.
[0012] In one alternative embodiment, the helmet body includes an outer rigid protective layer and an inner liner layer;
[0013] The first support, sliding component, and counterweight component are all fixed to the rigid protective layer;
[0014] The inner lining is a flexible pad that comes into direct contact with the head.
[0015] In one optional embodiment, the rigid protective layer has a plurality of strip-shaped slots;
[0016] The inner lining layer has several openings for ventilation; the positions of the openings correspond to the positions of the slots.
[0017] In an optional embodiment, an RGB camera or an ultra-wide-angle camera is respectively mounted on the second brackets located on both sides.
[0018] In an optional embodiment, an ultra-wide-angle camera or a microphone array with an integrated IMU pose sensor is respectively mounted on the second brackets on both sides.
[0019] Preferably, the second support is a serpentine support.
[0020] Preferably, the second bracket includes a spherical universal joint, a connecting rod, and a retainer; the universal joint is a universal joint with a damper, which maintains a fixed angle after manual adjustment of the position;
[0021] One end of the connecting rod is fixed to the universal joint, and the other end is equipped with a retainer for fixed connection with the sampling equipment.
[0022] In an optional embodiment, the first support is a three-axis gimbal.
[0023] In an optional embodiment, the sliding assembly includes a base, a guide bar, and a connecting portion;
[0024] The base is integrally formed with the rigid protective layer as the substrate; the guide strip is set on the base, and the connecting parts are set in pairs and are all installed on the guide strip, and slide relative to the guide strip;
[0025] The connecting part has a cylindrical structure with an outer shell. The middle of the outer shell has a notch for engaging with the guide strip. The bottom of the connecting part has a pressing part. When the pressing part is pressed upward, it causes the locking pin to extend and insert into the guide strip through the notch. The connecting part is fixed by the friction between the locking pin and the guide strip. The pressing part and the outer shell are magnetically attracted to each other.
[0026] When the pressing part moves downward, it causes the locking pin to disengage from the notch, at which point the connecting part slides back and forth relative to the guide bar.
[0027] Compared with the prior art, the present invention has the following beneficial technical effects:
[0028] This invention ensures the front-to-back balance of the helmet by installing a first bracket, a second bracket, and a counterweight assembly on the helmet. At the same time, different sampling devices, such as RealSense RGBD cameras, RGB cameras, ultra-wide-angle cameras, IMU pose sensors, or microphone arrays, are installed on the first and second brackets. This allows for the acquisition of images over a wide range and from multiple angles in a single operation, while also acquiring pose and sound information, thus achieving multimodal data acquisition. Attached Figure Description
[0029] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0030] Figure 1 This is a schematic diagram of the overall structure of an embodiment of the present utility model;
[0031] Figure 2 This is a schematic diagram of the back structure of an embodiment of the present utility model;
[0032] Figure 3 This is a schematic diagram of the unfolded structure of the helmet body in an embodiment of this utility model;
[0033] Figure 4 This is a schematic diagram of the connection structure between the sliding component and the second bracket in an embodiment of the present utility model;
[0034] Figure 5 This is one of the structural schematic diagrams of the connecting part in the embodiments of this utility model;
[0035] Figure 6 This is the second structural schematic diagram of the connecting part in an embodiment of this utility model.
[0036] Reference numerals: 1. Helmet body; 11. Hard protective layer; 111. Groove; 12. Inner liner; 121. Opening; 2. First bracket; 3. Counterweight assembly; 4. Sliding assembly; 41. Base; 42. Guide strip; 43. Connecting part; 431. Outer shell; 432. Pressing part; 433. Notch; 434. Locking pin; 5. Second bracket; 51. Universal joint; 52. Linkage rod; 6. Ultra-wide angle camera; 7. Strap; 8. RealSense RGBD camera. Detailed Implementation
[0037] To better understand the above technical solutions, the technical solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.
[0038] Example 1
[0039] like Figures 1-2 As shown, the present invention proposes a device for collecting multimodal body data, including a helmet body 1, a first support 2, a second support 5, a sliding component 4, and a counterweight component 3;
[0040] The first bracket 2 is set up in a group and located at the front of the helmet body 1; a RealSense RGBD camera 8 is installed on the first bracket 2 to capture image information of the front of the helmet body 1; RealSense is a real sense technology, and the image information collected by the camera is 3D image information with depth.
[0041] The sliding components 4 are symmetrically arranged on the left and right sides of the helmet body 1; the second bracket 5 is mounted on the sliding components 4 and slides relative to them along the sliding components 4; as shown Figure 4 As shown, the sliding assembly 4 includes a base 41, a guide bar 42, and a connecting part 43; the base 41 is integrally formed with the rigid protective layer 11 as a substrate; the guide bar 42 is disposed on the base 41, and the connecting parts 43 are arranged in pairs and are both installed on the guide bar 42, and slide relative to the guide bar 42; as shown Figure 5 As shown, the connecting part 43 has a cylindrical structure, with an outer shell 431. A notch 433 is provided in the middle of the outer shell 431 to engage with the guide strip 42. A pressing part 432 is provided at the bottom of the connecting part 43. When the pressing part 432 is pressed upwards, it causes the locking pin 434 to extend and insert into the guide strip 42 through the notch 433. The friction between the locking pin 434 and the guide strip 42 fixes the connecting part 43 in place. The pressing part 432 and the outer shell 431 maintain magnetic attraction. Figure 6As shown, when the pressing part 432 moves downward, it causes the locking pin 434 to disengage from the notch part 433. At this time, the connecting part 43 slides back and forth relative to the guide bar 42.
[0042] The sliding component 4 allows the second support 5 to move back and forth to a suitable position, facilitating the collection of different information.
[0043] In this embodiment, the counterweight assembly 3 is fixed to the rear end of the helmet body 1 to level the center of gravity of the entire helmet body 1 so that it is located at the geometric center of the helmet body 1; at the same time, the counterweight assembly 3 integrates a power supply module to supply power to the sampling equipment; the power supply module is electrically connected to each electrical device through wires to ensure the normal operation of each sampling device.
[0044] In this embodiment, a matching strap 7 for fixing is provided on the helmet body 1; the strap 7 is adjustable in tightness, and when in use, after the user puts on the helmet body 1, he can pull the strap 7 to tighten the helmet body 1.
[0045] Example 2
[0046] The present invention proposes a device for collecting multimodal embodied data, which differs from Embodiment 1 in that, Figure 3 As shown, the helmet body 1 includes an outer hard protective layer 11 and an inner liner 12;
[0047] The first support 2, sliding component 4, and counterweight component 3 are all fixed to the rigid protective layer 11; the inner lining layer 12 is a flexible padding layer that directly contacts the head. The rigid protective layer 11 has several strip-shaped slots 111; the inner lining layer 12 has several ventilation holes 121; the positions of the ventilation holes 121 correspond to the positions of the slots 111. The rigid protective layer 11 provides a base for the installation of each device and has a certain strength to withstand external impacts and is not easily damaged; while the inner lining layer 12 is a flexible padding layer, which is softer and more comfortable in direct contact with the head; at the same time, the rigid protective layer 11 and the inner lining layer 12 are respectively provided with strip-shaped slots 111 and ventilation holes 121, which allows users to easily breathe and sweat during use, making it more comfortable.
[0048] Example 3
[0049] This invention proposes a device for collecting multimodal embodied data. Unlike Embodiment 1, in this embodiment, RGB cameras or ultra-wide-angle cameras 6 are respectively mounted on the second supports 5 located on both sides. Furthermore, the second supports 5 are serpentine supports. In actual use, the RealSense RGBD camera 8 on the first support 2, which is a three-axis gimbal, provides image stabilization when collecting images with 3D depth of field, resulting in more stable images. The RGB cameras or ultra-wide-angle cameras 6 on the second supports 5 assist in collecting image information. The serpentine support design allows users to manually adjust the direction and angle of the second supports 5 to collect image information from different angles, thus increasing the range of image data collection.
[0050] Example 4
[0051] This utility model proposes a device for collecting multimodal embodied data. Unlike Embodiment 1, in this embodiment, an ultra-wide-angle camera 6 or a microphone array integrating an IMU pose sensor is respectively mounted on the second supports 5 on both sides. The second supports 5 include a spherical universal joint 51, a connecting rod 52, and a retainer; the universal joint 51 is a universal joint with a damper, which maintains a fixed angle after manual adjustment; one end of the connecting rod 52 is fixed to the universal joint 51, and the other end is equipped with a retainer for fixed connection with the sampling device.
[0052] In actual use, the RealSense RGBD camera 8 on the first bracket 2, which is a three-axis gimbal, plays a role in stabilizing the image when acquiring image information with 3D depth of field, making the image more stable; while the ultra-wide-angle camera 6 on the second bracket 5 can assist in acquiring image information; the microphone array with integrated IMU pose sensor can simultaneously acquire pose information and sound information; thus realizing multimodal data acquisition.
[0053] In the description of this application, it should be understood that the terms "upper", "lower", "left", "right", "front", "rear", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this utility model and simplifying the description, and do not indicate or imply that the device or unit referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this utility model.
[0054] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0055] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A device for collecting multimodal embodied data, characterized in that, It includes the helmet body (1), the first bracket (2), the second bracket (5), the sliding component (4), and the counterweight component (3); A first bracket (2) is set up at the front of the helmet body (1); a RealSense RGBD camera (8) is installed on the first bracket (2) to capture image information of the front of the helmet body (1); The sliding components (4) are symmetrically arranged on the left and right sides of the helmet body (1); the second bracket (5) is mounted on the sliding components (4) and slides relative to the sliding components (4); wherein, RGB cameras or ultra-wide-angle cameras (6) are respectively set on the second brackets (5) on both sides; or, ultra-wide-angle cameras (6) or microphone arrays with IMU pose sensors are respectively set on the second brackets (5) on both sides. The counterweight assembly (3) is fixed to the rear end of the helmet body (1) to level the center of gravity of the entire helmet body (1) so that it is located at the geometric center of the helmet body (1); The helmet body (1) is equipped with matching straps (7) for fixing.
2. The device for collecting multimodal embodied data according to claim 1, characterized in that, The counterweight assembly (3) integrates a power supply module for powering the sampling equipment; the power supply module is electrically connected to each electrical device via wires.
3. The device for collecting multimodal embodied data according to claim 1, characterized in that, The helmet body (1) includes an outer hard protective layer (11) and an inner liner (12). The first support (2), the sliding component (4) and the counterweight component (3) are all fixed on the rigid protective layer (11); The inner lining (12) is a flexible pad that is in direct contact with the head.
4. The device for collecting multimodal embodied data according to claim 3, characterized in that, The hard protective layer (11) has several strip-shaped slots (111). The inner lining (12) has several openings (121) for ventilation; the positions of the openings (121) correspond to the positions of the slots (111).
5. The apparatus for acquiring multimodal embodied data according to any one of claims 1-4, characterized in that, The second support (5) is a serpentine support.
6. The apparatus for acquiring multimodal embodied data according to any one of claims 1-4, characterized in that, The second bracket (5) includes a spherical universal joint (51), a connecting rod (52) and a retainer; the universal joint (51) is a universal joint with a damper, which can maintain a fixed angle after manual adjustment of the position; One end of the connecting rod (52) is fixed to the universal joint (51), and the other end is equipped with a retainer for fixed connection with the sampling equipment.
7. The device for collecting multimodal embodied data according to claim 1, characterized in that, The first support (2) is a three-axis gimbal.
8. The device for collecting multimodal embodied data according to claim 3, characterized in that, The sliding component (4) includes a base (41), a guide bar (42), and a connecting part (43); The base (41) is integrally formed with the hard protective layer (11) as the base material; the guide strip (42) is set on the base (41), and the connecting parts (43) are arranged in pairs and are all installed on the guide strip (42) and slide relative to the guide strip (42); The connecting part (43) has a cylindrical structure and an outer shell (431). The middle part of the outer shell (431) is provided with a notch (433) that engages with the guide strip (42). The bottom of the connecting part (43) is provided with a pressing part (432). When the pressing part (432) is pressed upward, it drives the locking pin (434) to extend and insert into the guide strip (42) through the notch (433). The connecting part (43) is fixed by the friction between the locking pin (434) and the guide strip (42). The pressing part (432) and the outer shell (431) are magnetically attracted to each other. When the pressing part (432) moves downward, it causes the locking pin (434) to disengage from the notch (433), and at this time the connecting part (43) slides back and forth relative to the guide bar (42).