Integrated facial motion capture module, head-mounted device accessory, and system thereof

Abstract

The application relates to the technical field of facial motion capture, and discloses an integrated facial dynamic capture module, a head-mounted device accessory and a system thereof, wherein the integrated facial dynamic capture module comprises a bottom shell and a front cover, the bottom shell is connected with the front cover, a control mainboard and a camera are arranged in the bottom shell, the camera is electrically connected with the control mainboard, a first interface is arranged on the side, away from the camera, of the control mainboard, and the first interface is used for electrically connecting with a head-mounted device through a data line; the camera and the related control unit thereof are decoupled from the VR head-mounted device to form an independent and detachable module, the problem of poor flexibility and low adaptability caused by the fixed connection structure in the prior art is solved, power supply connection and data transmission are realized through the first interface, the facial capture module does not need to be internally provided with a battery, the weight is reduced, the light-weight use requirement is met, and the head-mounted device accessory and the head-mounted device system are applicable to the above facial capture dynamic capture module.

Description

Technical Field

[0001] This application relates to the technical field of facial motion capture, and in particular to an integrated facial motion capture module, head-mounted display device accessories and systems thereof. Background Technology

[0002] Chinese patent publication number CN107491165A, entitled: A VR glasses facial 3D image, planar image capture and gesture capture system, specifically discloses that a camera n is set on the edge of the lower frame of the VR glasses, and the camera n is set at the center point between the left and right sides, and the image capture direction is facing the face; the camera n can obtain real-time changes in the shape of the mouth and real-time movement changes of the lower and upper jaws. However, the camera and VR headset use a fixed connection structure, forming an inseparable whole. This integrated design makes it impossible to quickly disassemble and replace according to user needs, which greatly limits the application scenarios of the device. For example, users cannot share a high-performance capture module between different models of VR headsets, or quickly replace the device when it is damaged, making it difficult to meet the growing demand for flexibility and adaptability.

[0003] Therefore, there is an urgent need for integrated facial motion capture modules, head-mounted display accessories, and related systems to solve the above problems.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Utility Model Content

[0005] The main purpose of this application is to provide an integrated facial motion capture module, head-mounted display accessories and system, to solve the problem that existing facial capture modules cannot be disassembled and adapted to various head-mounted display devices, thus limiting their scope of application.

[0006] To achieve the above objectives, the integrated facial motion capture module provided in this application includes a bottom shell and a front cover. The bottom shell is connected to the front cover. A control motherboard and a camera are disposed in the bottom shell. The camera is electrically connected to the control motherboard. A first interface is disposed on the side of the control motherboard away from the camera. The first interface is used to electrically connect to the head-mounted display device via a data cable.

[0007] As a preferred embodiment of this application, it also includes infrared fill lights, and a plurality of infrared fill lights are provided on the circumferential edge of the control motherboard, and the plurality of infrared fill lights are electrically connected to the control motherboard.

[0008] As a preferred embodiment of this application, it further includes a connecting portion, which is provided on the top of the bottom shell, and the connecting portion is used for detachable connection with a bracket.

[0009] As a preferred embodiment of this application, it also includes a limiting groove, which is composed of multiple limiting plates arranged around the bottom shell, and the camera is disposed in the limiting groove.

[0010] As a preferred embodiment of this application, it also includes a shock-absorbing pad, which is disposed between the limiting groove and the camera, and the shock-absorbing pad is used to isolate mechanical vibrations generated during movement.

[0011] As a preferred embodiment of this application, the central area of ​​the front cover has a gradient raised structure and a through hole, the camera is positioned directly opposite the through hole, and the front cover is made of a high light transmittance material.

[0012] As a preferred embodiment of this application, it further includes a mounting groove, which protrudes from the bottom shell and penetrates the bottom shell, with the first interface located in the mounting groove.

[0013] As a preferred embodiment of this application, the camera is a miniature wide-angle camera.

[0014] A head-mounted display accessory includes a data cable, a stand, and the aforementioned integrated facial motion capture module; wherein, The data cable has one end connected to the first interface of the integrated facial motion capture module and the other end connected to the second interface of the head-mounted display device. The first interface is used for power supply and data transmission. The bracket is provided with a first mounting part and a second mounting part. The first mounting part is detachably connected to the integrated facial motion capture module, and the second mounting part is used to connect to the head-mounted display device.

[0015] A head-mounted display system includes the aforementioned head-mounted display accessories and the head-mounted display itself; wherein... The second mounting part is detachably connected to the head-mounted display device; The head-mounted display device internally includes a power module, a data processing module, and a wireless transmission module. The second interface of the head-mounted display device is internally connected to the power module and the data processing module, and externally connected to the first interface via the data cable.

[0016] The integrated facial motion capture module provided in this application decouples the camera and its related control unit from the VR headset to form an independent, detachable module. This solves the problems of poor flexibility and low adaptability caused by the fixed connection structure in the prior art. At the same time, the power supply connection and data transmission are realized through the first interface. The facial capture module does not need to have a built-in battery, which helps to reduce the weight and meet the requirements of lightweight use. Attached Figure Description

[0017] Figure 1 This is a three-dimensional structural diagram of the integrated facial motion capture module in one embodiment of this application; Figure 2 This is a first exploded view of the integrated facial motion capture module in one embodiment of this application; Figure 3 This is a second exploded view of the integrated facial motion capture module in one embodiment of this application.

[0018] Explanation of reference numerals in the attached figures: 1. Bottom shell; 2. Front cover; 3. Control motherboard; 4. Camera; 5. First interface; 6. Infrared fill light; 7. Connecting part; 8. Limiting plate; 9. Shock-absorbing pad; 10. Mounting slot. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] Furthermore, descriptions using terms such as "first" and "second" in this application are for descriptive purposes only (e.g., to distinguish identical or similar elements) and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, technical solutions from different embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If a combination of technical solutions is contradictory or impossible to implement, such a combination should be considered nonexistent and not within the scope of protection claimed in this application.

[0021] Please refer to Figure 1 , Figure 2 , Figure 3 In one embodiment, the integrated facial motion capture module includes a bottom shell 1 and a front cover 2. The bottom shell 1 is connected to the front cover 2. A control motherboard 3 and a camera 4 are disposed in the bottom shell 1. The camera 4 is electrically connected to the control motherboard 3. A first interface 5 is disposed on the side of the control motherboard 3 away from the camera 4. The first interface 5 is used to electrically connect to the head-mounted display device through a data cable.

[0022] Preferably, the control motherboard 3 is an integrated ESP32-S3 dual-core microcontroller (240MHz), with 520KB SRAM and 8MB PSRAM, which is specifically used for real-time acquisition, preprocessing and transmission of image data. Furthermore, the control motherboard 3 has a through hole in the middle through which the camera 4 passes. The control motherboard 3 is fixed to the bottom shell 1 by screws and presses against the camera base to fix it. Preferably, the first interface 5 is a USB-C interface, which can be directly connected to the head-mounted display device to realize power supply and data transmission. With this connection method, the face patch module does not need to have a built-in battery, and its overall weight is effectively reduced, making it more suitable for lightweight use.

[0023] Specifically, camera 4 is a miniature wide-angle camera with a 160° wide field of view and a resolution of 240×240 pixels. Its field of view can completely cover the user's mouth, jaw, cheeks and other key facial expression areas, ensuring that all subtle muscle movements are captured without omission.

[0024] Specifically, the head-mounted display device is an Android-based all-in-one VR head-mounted display (Meta Quest series, Pico series, etc.). During use, the facial motion capture module continuously captures muscle movement images of the user's mouth, jaw, and cheeks from a 160° wide-angle perspective using a built-in miniature wide-angle camera 4. The ESP32-S3 microcontroller then performs real-time preprocessing on the captured image data and transmits the processed data at high speed to the Android-based VR headset via a USB-C interface. Finally, the headset system completes real-time modeling and driving of facial expressions. By decoupling the camera 4 and its related control unit from the VR headset to form an independent, detachable module, the module solves the problems of poor flexibility and low adaptability caused by the fixed connection structure in the existing technology. Utilizing an integrated hardware design and direct power supply transmission mode, it avoids the weight burden of the built-in battery and significantly improves the device's lightweight design and wearing comfort.

[0025] Specifically, please refer to Figure 2 Based on the above embodiments, it also includes infrared fill lights 6. Multiple infrared fill lights 6 are provided on the circumferential edge of the control motherboard 3, and the multiple infrared fill lights 6 are electrically connected to the control motherboard 3.

[0026] It is understandable that the infrared fill light 6 specifically adopts an LED infrared fill light 6. When in use, the LED infrared fill light 6 array arranged circumferentially on the control motherboard 3 is activated first, emitting infrared light with a wavelength of 850nm to uniformly illuminate the facial area. By setting the LED infrared fill light 6, the influence of changes in ambient visible light (such as the intensity of indoor lighting and sunlight interference) on imaging is effectively overcome, ensuring that a stable, uniform and glare-free lighting environment can be provided for the camera 4 under any lighting conditions, laying the foundation for high-quality image acquisition.

[0027] Further, please refer to Figure 1 , Figure 2 Based on the above embodiments, a connecting part 7 is also included. The top of the bottom shell 1 is provided with a connecting part 7, which is used to detachably connect with a bracket.

[0028] Specifically, in this embodiment, the structure of the connecting part 7 is a hook, and the number of hooks is at least one. In this embodiment, two are taken as an example. The openings on the two hooks are respectively facing the bottom shell 1 and the other facing the front cover 2. The hook is placed in the groove of the bracket, and the hand-tightened bolt passes through the opening and is connected to the nut on the bracket to realize the connection between the face capture module and the bracket. After installation, the installation angle of the face capture module can be adjusted by tightening or loosening the bolt. In addition, the connecting part 7 at the top of the bottom shell 1 of the face capture module can be configured to achieve a quick magnetic, snap-on or threaded detachable connection with the matching bracket. The above settings ensure the stability and angle adjustment of the facial capture module during the capture process, while also enabling convenient disassembly, assembly, and portable storage, allowing for flexible adaptation to various installation scenarios.

[0029] Specifically, please refer to Figure 2 Based on the above embodiments, a limiting groove is also included. The limiting groove is composed of multiple limiting plates 8, which are arranged around the bottom shell 1. The camera 4 is disposed in the limiting groove.

[0030] It is understandable that by setting a limiting groove consisting of multiple limiting plates 8 surrounding the bottom shell 1 and precisely fixing the camera 4 in the groove, the camera 4 is effectively prevented from shifting or loosening during installation, use or accidental collision, ensuring that its optical center is always precisely aligned with the preset capture area; thus ensuring the stability and consistency of image acquisition, and at the same time, avoiding the loss of facial data due to viewing angle deviation.

[0031] Specifically, please refer to Figure 2 Based on the above embodiments, a shock-absorbing pad 9 is also included. A shock-absorbing pad 9 is provided between the limiting groove and the camera 4. The shock-absorbing pad 9 is used to isolate mechanical vibrations generated during movement.

[0032] It is understandable that the shock-absorbing pad 9 is made of high-damping EVA material. By setting the shock-absorbing pad 9 between the limiting groove and the camera 4, it can actively absorb and isolate high-frequency mechanical vibrations transmitted from the user's head movement or external sources, significantly reducing the jitter interference caused by vibration in the image acquisition process. This ensures that the facial images captured by the camera 4 are more stable and clear, improving the quality of the raw data from the source. This provides a key guarantee for achieving high-precision, jitter-free facial expression restoration in the future, enhancing the reliability and capture accuracy of the system in dynamic usage scenarios. At the same time, the elastic deformation of the shock-absorbing pad can eliminate the installation impact caused by dimensional errors in manufacturing.

[0033] Specifically, please refer to Figure 2 Based on the above embodiments, the central area of ​​the front cover 2 has a gradient raised structure and a through hole, with the camera 4 positioned directly opposite the through hole. The front cover 2 is made of a high light transmittance material.

[0034] It is understandable that the gradient protrusion structure in the center of the front cover 2 provides space for the camera 4 and the multiple infrared fill lights 6 on its circumferential edge, so that they will not interfere with the inner surface of the front cover 2; at the same time, the high light transmittance of the front cover 2 material ensures that infrared light can be transmitted efficiently and uniformly.

[0035] Specifically, please refer to Figure 2 Based on the above embodiments, it also includes a mounting groove 10, which is protruding in the bottom shell 1 and penetrates through the bottom shell 1. The first interface 5 is located in the mounting groove 10.

[0036] It is understandable that by providing a through mounting groove 10 on the bottom shell 1 and securing the USB-C interface in this groove, the mounting groove 10 provides precise insertion and removal guidance and circumferential support for the data cable plug, effectively resisting lateral stress and accidental pulling from the cable during daily use, preventing the interface from becoming loose; and improving the long-term reliability of the power supply and data transmission connection between the module and the head-mounted display device.

[0037] A head-mounted display accessory includes a data cable, a stand, and the aforementioned integrated facial motion capture module; wherein, The data cable has one end connected to the first interface 5 of the integrated facial motion capture module and the other end connected to the second interface on the head-mounted display device. The first interface 5 is used for power supply and data transmission. The bracket is provided with a first mounting part and a second mounting part. The first mounting part is detachably connected to the integrated facial motion capture module, and the second mounting part is used to connect to the head-mounted display device.

[0038] It is understandable that the data cable specifically adopts a USB Type-C data cable that supports USB 2.0 data transmission. By using the first interface 5, the data cable and the second interface, a power supply connection and data processing are established between the facial motion capture module and the head-mounted display device, which can realize quick disassembly and assembly and adapt to other head-mounted display devices. Furthermore, the shape of the bracket can be designed as an arc structure according to the shape of the head-mounted display device. This arc structure is adapted to the mounting surface of the head-mounted display device. Preferably, a through hole can be opened in the middle of the arc-shaped bracket, and a protrusion can be set on the head-mounted display device. After connecting the through hole and the protrusion, the bracket and the head-mounted display device can be connected. The direct end is the first mounting part. The structure of the first mounting part can be that a groove is opened at one end of the bracket. A hand-tightened bolt is passed through the hole on the bracket and the opening of the connecting part 7 in sequence, and the threaded part of the hand-tightened bolt is connected to the nut on the bracket to realize the connection between the face capture module and the bracket. After installation, the installation angle of the face capture module can be adjusted by tightening or loosening the bolt.

[0039] This connection method allows for quick assembly and disassembly of the facial motion capture module, facilitating maintenance. Furthermore, due to the universality of its interface protocol, it can flexibly adapt to various head-mounted display devices, improving accessory compatibility.

[0040] A head-mounted display system includes the aforementioned head-mounted display accessories and a head-mounted display device; wherein, The second mounting section is detachably connected to the head-mounted display device; The head-mounted display device has a power module, a data processing module and a wireless transmission module inside. The second interface of the head-mounted display device is internally connected to the power module and the data processing module, and externally connected to the first interface 5 through a data cable. Specifically, the first interface and the second interface establish a proprietary protocol (non-UVC type) data link based on USB 2.0 to carry JPEG format image frames; the data processing module runs a dedicated application to parse the proprietary protocol and cache and convert the image frames; and the wireless transmission module forwards the processed data to external devices via 5GHz Wi-Fi.

[0041] In summary, it can be understood that the integrated facial motion capture module provided in this application is electrically connected to the head-mounted display device through a USB-C interface. The power module built into the head-mounted display device provides 5V DC power to the facial motion capture module through the USB-C interface, and the total operating current of the facial motion capture module is no more than 500mA. By setting multiple infrared fill lights 6 around the edge of the control motherboard 3, and ensuring that the light can be evenly projected onto the user's face through the high light transmittance front cover 2, sufficient and uniform illumination is provided for the camera 4, ensuring that facial image features can be clearly and stably captured in any lighting environment. The control motherboard 3 of the facial motion capture module is based on the ESP32-S3 controller. It is configured to control the camera 4 to capture facial images at a maximum frame rate of 60fps (30Hz by default). The captured image data is preprocessed and compressed locally in JPEG format. The processed image data is encapsulated and transmitted through a proprietary protocol (non-UVC class) based on USB 2.0. The protocol uses custom frame boundaries and metadata fields. The dedicated application built into the head-mounted display device is responsible for parsing the proprietary protocol and completing data caching and format conversion. The data processing module within the head-mounted display (HMD) runs a dedicated application program to receive and parse JPEG image data carried by the proprietary protocol, performing data format conversion, caching, and optional feature preprocessing. The processed data is then forwarded via the HMD's wireless transmission module. This wireless transmission module is preferably a 5GHz Wi-Fi module supporting the IEEE 802.11ac standard, with a typical transmit power of approximately 200mW, used to stably transmit data to an external processing device (such as a PC) for neural network recognition. This avoids the constraints of long-distance wired connections and mitigates the interference susceptibility of traditional 2.4GHz links.

[0042] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0043] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An integrated facial motion capture module, characterized in that, The device includes a bottom shell and a front cover. The bottom shell is connected to the front cover. A control motherboard and a camera are disposed in the bottom shell. The camera is electrically connected to the control motherboard. A first interface is disposed on the side of the control motherboard away from the camera. The first interface is used to electrically connect to the head-mounted display device via a data cable.

2. The integrated facial motion capture module according to claim 1, characterized in that, It also includes infrared fill lights, and multiple infrared fill lights are provided on the circumferential edge of the control motherboard, and the multiple infrared fill lights are electrically connected to the control motherboard.

3. The integrated facial motion capture module according to claim 1, characterized in that, It also includes a connecting part, which is provided on the top of the bottom shell, and the connecting part is used for detachable connection with a bracket.

4. The integrated facial motion capture module according to claim 1, characterized in that, It also includes a limiting groove, which is composed of multiple limiting plates arranged around the bottom shell, and the camera is disposed in the limiting groove.

5. The integrated facial motion capture module according to claim 4, characterized in that, It also includes a shock-absorbing pad, which is provided between the limiting groove and the camera. The shock-absorbing pad is used to isolate mechanical vibrations generated during movement.

6. The integrated facial motion capture module according to claim 1, characterized in that, The center area of ​​the front cover has a gradient raised structure and a through hole. The camera is positioned directly opposite the through hole. The front cover is made of a high light transmittance material.

7. The integrated facial motion capture module according to claim 1, characterized in that, It also includes a mounting groove, which protrudes into the bottom shell and penetrates through the bottom shell, with the first interface located in the mounting groove.

8. The integrated facial motion capture module according to claim 6, characterized in that, The camera is a miniature wide-angle camera.

9. A head-mounted display accessory, characterized in that, Includes a data cable, a stand, and an integrated facial motion capture module as described in any one of claims 1-8; in, The data cable has one end connected to the first interface of the integrated facial motion capture module and the other end connected to the second interface of the head-mounted display device. The first interface is used for power supply and data transmission. The bracket is provided with a first mounting part and a second mounting part. The first mounting part is detachably connected to the integrated facial motion capture module, and the second mounting part is used to connect to the head-mounted display device.

10. A head-mounted display system, characterized in that, Includes the head-mounted display device accessory as described in claim 9 and the head-mounted display device; wherein, The second mounting part is detachably connected to the head-mounted display device; The head-mounted display device internally includes a power module, a data processing module, and a wireless transmission module. The second interface of the head-mounted display device is internally connected to the power module and the data processing module, and externally connected to the first interface via the data cable.

Images