A deep learning-based facial motion capture method

Abstract

This invention discloses a deep learning-based facial motion capture method, belonging to the field of facial capture technology, including the following steps: S1, setting up a dynamic face capture device to collect data from various aspects of the face and construct a multi-faceted face dataset; S2, integrating the dynamic face capture device to simultaneously output 3-5 video streams and RGB and depth information; S3, using a multi-view stereo matching algorithm to generate a dense 3D point cloud; calculating the multi-view error, and by continuously injecting gas into the internal airbag, the internal airbag expands and pushes the outer supporting bladder to tilt, thereby causing the camera to tilt. When multiple cameras tilt, they can capture facial data from the sides of the human face. The overall side data acquisition and capture is based on the same face, reducing the need for manual stitching and adjustment in multi-camera situations.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face capture, in particular to a face motion capture method based on deep learning. BACKGROUND

[0002] In the field of film and television animation production, generating character animation by capturing real face motion is a key technical link to improve the realism and expressiveness of works. Traditional face motion capture technology mainly relies on optical markers or mechanical devices, records the motion trajectory of key points on the face through a multi-angle camera array or sensor network, and then drives the expression changes of digital characters. The breakthrough of deep learning technology brings revolutionary changes to face motion capture. Based on the end-to-end solution of convolutional neural network (CNN) and recurrent neural network (RNN), the face video stream is collected in real time by infrared camera or ordinary RGB camera, and the pre-trained model is used to automatically extract key feature points and generate three-dimensional expression parameters, which solves the problems in the traditional capture process.

[0003] In the existing patent, it is pointed out that the current mainstream methods can be divided into two-dimensional data-based and three-dimensional data-based. The former uses optical lens to understand human facial expressions and actions through algorithms, such as Faceware's helmet-style single-camera face motion capture system. The advantages of this method are low cost, easy to obtain, and convenient to use, and the disadvantages are lower capture accuracy compared with other methods; the latter obtains two-dimensional data through optical lens, and obtains depth information through additional means or devices, such as multi-camera and structured light. This method has fast processing speed and high accuracy, but requires additional depth acquisition devices. It solves the problem of operation efficiency of traditional capture by outputting mixed shape coefficients and applying the coefficients to virtual images for real-time rendering of expressions. However, the overall process is still based on single-camera image capture, and the overall process is only supplemented by algorithms to complete the missing images. However, in the field of film and television animation, especially in 3D films, the micro-expression of human face will still appear distortion phenomenon when completing, which affects the output content, and still needs a lot of manual calibration in the later stage. Moreover, the existing multi-camera is still based on the current angle to shoot the human face directly, and the capture of muscle conditions on the side of the human face is still poor. Therefore, a face motion capture method based on deep learning is proposed. SUMMARY

[0004] The present application aims to provide a face motion capture method based on deep learning to solve the problems mentioned in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: a face motion capture method based on deep learning, comprising the following steps:

[0006] S1. Set up a dynamic face capture device, collect data from all sides of the face through the dynamic face capture device, and construct a multi-faceted face dataset;

[0007] S2. Integrates dynamic face capture device to synchronously output one video stream and RGB information and depth information;

[0008] S3. Use a multi-view stereo matching algorithm to generate dense 3D point clouds;

[0009] S4. Reconstruct the micro-expression-based hybrid shape model based on dense D-point cloud sets;

[0010] S5. Generate a multi-view fusion facial action recognition network through a hybrid shape model and calculate the multi-view error;

[0011] S6. By using a facial motion recognition network and calculating the numerical value of multi-view error, the skeletal topology of different virtual characters in the video is obtained through a deformation transfer algorithm.

[0012] Preferably, the dynamic face capture device in S1 includes a mounting box. A rotating plate is rotatably connected inside the mounting box. A central support capsule and multiple peripheral support capsules are fixedly connected to the outside of the rotating plate, with the peripheral support capsules surrounding the central support capsule. A mounting plate is fixedly connected to the side of the central and peripheral support capsules away from the rotating plate. A camera is installed inside the mounting plate. An air pump is installed inside the mounting box. The air outlet of the air pump is connected to a delivery pipe. A rotary joint is installed at the end of the delivery pipe away from the air pump. The rotating outlet of the rotary joint penetrates the outer wall of the rotating plate and communicates with the central support capsule. Multiple second electrically controlled metering valves are connected to the outside of the central support capsule. The multiple second electrically controlled metering valves are respectively connected to the multiple peripheral support capsules. A micro drive motor is installed inside the mounting box. A friction wheel is installed on the output shaft of the micro drive motor, and the friction wheel is in contact with the outside of the rotating plate.

[0013] Preferably, each of the plurality of peripheral support bladders has an internally formed internal pushing air bladder, the external of which is connected to a first electrically controlled metering valve, the first electrically controlled metering valve being connected to the air outlet of a second electrically controlled metering valve, the internal of the central support bladder being fixedly connected to a second electromagnet, and the internal of the peripheral support bladder being fixedly connected to a first electromagnet.

[0014] Preferably, the outer part of the inner push air bag is integrally formed with an outwardly expanding reflective bag, the outwardly expanding reflective bag penetrates the outer wall of the outer peripheral bearing bag body, the air inlet of the outwardly expanding reflective bag is communicated with a third metering electric control valve, the air inlet of the third metering electric control valve is communicated with the inner push air bag, the outer part of the outwardly expanding reflective bag is integrally formed with a reflective layer, and the outer parts of the outwardly expanding reflective bag, the central bearing bag body, the outer peripheral bearing bag body and the inner push air bag are all communicated with pressure relief metering valves.

[0015] Preferably, the outer part of the reflective layer of the outwardly expanding reflective bag is attached with an electrochromic film, and the electrochromic film is externally connected with an electrification wire which is electrically connected with an external power source.

[0016] Preferably, the bottom of the mounting box is fixedly connected with a telescopic rod, and the bottom of the telescopic rod is fixedly connected with a base.

[0017] Preferably, the outer part of the conveying pipe is communicated with an external conveying valve, one end of the external conveying valve away from the conveying pipe penetrates the outer walls of the rotating plate and the telescopic rod and is communicated with an extension pipe, the inner part of the telescopic rod is fixedly connected with an inner push multi-stage dynamic bag body, and the air outlet of the extension pipe is communicated with the inner push multi-stage dynamic bag body.

[0018] Preferably, the inner part of the base is provided with a sliding groove, the inner part of the sliding groove is slidably connected with a T-shaped sliding block, and the bottom of the T-shaped sliding block is fixedly connected with a fastening belt.

[0019] Preferably, the bottom of the base is fixedly connected with a suction disc.

[0020] Preferably, the outer part of the central bearing bag body is fixedly connected with a plurality of second strain gauges, the outer parts of the inner push air bag and the outer peripheral bearing bag body are both fixedly connected with a plurality of first strain gauges, the first strain gauges and the second strain gauges are used to generate resistance changes when the central bearing bag body and the outer peripheral bearing bag body are inflated and extended, the angle position of the camera is calculated through the resistance changes and the inflation amount, and the calculation formula is:

[0021] ;

[0022] wherein, is the inclination angle of the outer peripheral bearing bag body;

[0023] is the original length of the outer peripheral bearing bag body;

[0024] is the initial resistance of the first strain gauge;

[0025] is the air bag deformation coefficient;

[0026] is the gas volume entering the inner push air bag;

[0027] is the resistance change amount of the first strain gauge,

[0028] The formula of the relationship between the gas volume and the extension amount of the capsule is: , wherein, is the deformation amount, is the proportional coefficient, is the driving amount, i.e., the input variable;

[0029] After the above relationship variable is obtained, it is substituted into the geometric tilt model, and the specific formula of the geometric tilt model is:

[0030] , wherein, is the cosine value of the tilt angle of the outer peripheral bearing capsule

[0031] is the extension amount of the front end of the capsule.

[0032] Compared with the prior art, the beneficial effects of the present application are:

[0033] In the present application, multiple shooting in the same direction can be realized by the central bearing capsule and the plurality of outer peripheral bearing capsules, multiple video stream outputs are realized, and at the same time, the plurality of outer peripheral bearing capsules or the central bearing capsule can be inflated and extended to different degrees by injecting gas into the inside of the outer peripheral bearing capsule or the central bearing capsule, so as to drive the plurality of cameras to approach or move away from the human face in the physical distance, and the human face data is collected at different distances.

[0034] In the present application, the gas injected into the inside of the inner push capsule is continuously pushed inwards, so that the inner push capsule can push the outer peripheral bearing capsule to tilt in the case of expansion, thereby driving the camera to tilt, and the plurality of cameras can capture the side face data of the human face in the case of tilting, and the side face data of the same face is collected and captured as a whole, thereby reducing the phenomenon that manual multiple splicing and adjustment are required in the case of multiple positions. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flow structure schematic diagram in the embodiment of the present application;

[0036] Figure 2 is a structure schematic diagram of the dynamic face capturing device in the embodiment of the present application;

[0037] Figure 3 is a gas pump structure schematic diagram of the dynamic face capturing device in the embodiment of the present application;

[0038] Figure 4 ​​The schematic diagram of the internal structure of the peripheral bearing capsule of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0039] Figure 5 The schematic diagram of the sectional structure of the central bearing capsule of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0040] Figure 6 The schematic diagram of the three-dimensional structure of the central bearing capsule of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0041] Figure 7 The schematic diagram of the inclined state structure of the peripheral bearing capsule of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0042] Figure 8 The schematic diagram of the inflation state structure of the outwardly expanding light reflecting capsule of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0043] Figure 9 The schematic diagram of the structure of the electrochromic film of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0044] Figure 10 The schematic diagram of the sectional structure of the telescopic rod of the dynamic face capturing device in the embodiment of the present application is shown in the figure.

[0045] In the figure: 100, mounting box; 101, central bearing capsule; 102, peripheral bearing capsule; 103, camera; 104, rotating plate; 105, air pump; 106, conveying pipe; 107, rotating joint; 108, micro drive motor; 109, first strain gauge; 110, mounting plate; 200, inner push air bag; 201, first electric control metering valve; 202, second electric control metering valve; 203, first electromagnet sheet; 204, second electromagnet sheet; 205, second strain gauge; 300, outwardly expanding light reflecting capsule; 301, third metering electric control valve; 302, pressure relief metering valve; 400, electrochromic film; 500, telescopic rod; 501, base; 600, T-shaped sliding block; 601, fastening belt; 700, suction cup; 800, external conveying valve; 801, extension pipe; 802, inner push multi-stage dynamic capsule. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0047] Embodiment one, as Figure 1As shown, this application discloses a deep learning-based facial motion capture method, which includes the following steps:

[0048] S1. Set up a dynamic face capture device, collect data from all sides of the face through the dynamic face capture device, and construct a multi-faceted face dataset;

[0049] S2. Integrates dynamic face capture device to simultaneously output 3-5 video streams and RGB and depth information;

[0050] S3. Use a multi-view stereo matching algorithm to generate dense 3D point clouds;

[0051] S4. Reconstruct the micro-expression-based hybrid shape model based on dense 3D point cloud sets;

[0052] S5. Generate a multi-view fusion facial action recognition network through a hybrid shape model and calculate the multi-view error;

[0053] S6. By using a facial motion recognition network and calculating the numerical value of multi-view error, the skeletal topology of different virtual characters in the video is obtained through a deformation transfer algorithm.

[0054] like Figures 1-7 As shown, the dynamic face capture device in S1 includes a mounting box 100. A rotating plate 104 is rotatably connected inside the mounting box 100. A central support capsule 101 and multiple peripheral support capsules 102 are fixedly connected to the outside of the rotating plate 104. The multiple peripheral support capsules 102 surround the central support capsule 101. A mounting plate 110 is fixedly connected to the side of the central support capsule 101 and the peripheral support capsules 102 away from the rotating plate 104. A camera 1 is installed inside the mounting plate 110. 03. An air pump 105 is installed inside the mounting box 100. The air outlet of the air pump 105 is connected to a delivery pipe 106. A rotary joint 107 is installed at the end of the delivery pipe 106 away from the air pump 105. The rotary outlet of the rotary joint 107 passes through the outer wall of the rotating plate 104 and is connected to the central support bladder 101. Multiple second electrically controlled metering valves 202 are connected to the outside of the central support bladder 101. The multiple second electrically controlled metering valves 202 are respectively connected to multiple peripheral support bladders 102.

[0055] Specifically, during use, when data collection of the faces of film and television workers is required, the central support capsule 101 and the peripheral support capsule 102 can be placed in front of the human face. After placement, by activating the camera 103, multiple cameras 103 can be used to capture images of the human face and output multiple video streams. During use, gas can be continuously injected into the delivery tube 106 by activating the air pump 105. When gas is continuously injected into the delivery tube 106, the delivery tube 106 will deliver the gas to the rotary joint 107. When the gas is delivered to the rotary joint 107, it will be delivered to the central support capsule 101. When the gas enters the central support capsule 101, the central support capsule 101 will expand. When the central support capsule 101 expands, it can drive the camera 103 to extend forward, thereby adjusting the shooting distance.

[0056] Furthermore, by activating multiple second electronically controlled metering valves 202, the gas inside the central support bladder 101 can be injected into the outer support bladder 102. When the gas is injected into the outer support bladder 102, the multiple outer support bladders 102 will also expand. When the outer support bladder 102 expands, the expansion of the outer support bladder 102 will cause the mounting plate 110 and the camera 103 to extend forward, thereby enabling multiple shots of close-up or long-range focus when the whole is close to or far from the face during the shooting process, reducing the problem of blurring that may occur with multi-view cameras, and realizing physical distance adjustment.

[0057] like Figures 1-2 As shown, a telescopic rod 500 is fixedly connected to the bottom of the mounting box 100, and a base 501 is fixedly connected to the bottom of the telescopic rod 500.

[0058] Specifically, during use, when it is necessary to align the position of the mounting box 100 with the face of the person being photographed, the height of the mounting box 100 can be adjusted by adjusting the telescopic length of the telescopic rod 500.

[0059] like Figures 2-3 As shown, a sliding groove is provided inside the base 501, and a T-shaped slider 600 is slidably connected inside the sliding groove. A fastening band 601 is fixedly connected to the bottom of the T-shaped slider 600, and a suction cup 700 is fixedly connected to the bottom of the base 501.

[0060] Specifically, when performing wire stunts and other similar projects that require capturing the faces of the person being filmed during the wire stunt process, the T-shaped slider 600 can be inserted into the sliding groove, and the T-shaped slider 600 can be fixed to the designated wire crossbar by the fastening strap 601. This fixes and limits the position of the telescopic rod 500 and the mounting box 100. Furthermore, the suction cup 700 ensures the stability between the base 501 and the placement surface when the base 501 is placed in the designated position.

[0061] like Figure 10 As shown, an external delivery valve 800 is connected to the outside of the delivery pipe 106. The end of the external delivery valve 800 away from the delivery pipe 106 passes through the outer wall of the rotating plate 104 and the telescopic rod 500 and is connected to an extension pipe 801. An internally pushed multi-stage moving bladder 802 is fixedly connected inside the telescopic rod 500. The air outlet of the extension pipe 801 is connected to the internally pushed multi-stage moving bladder 802.

[0062] Specifically, during use, when it is necessary to actively adjust the telescopic distance of the telescopic rod 500, gas can be continuously injected into the delivery pipe 106 by starting the air pump 105 and closing the rotary joint 107. With the rotary joint 107 closed, the gas will be continuously delivered to the external delivery valve 800. When the gas is delivered to the external delivery valve 800, it will be delivered to the internal multi-stage moving bladder 802 through the extension pipe 801. When gas is continuously entering the internal multi-stage moving bladder 802, the internal multi-stage moving bladder 802 will expand. When the internal multi-stage moving bladder 802 expands, it will push the telescopic rod 500, thereby causing the multi-section telescopic sections of the telescopic rod 500 to extend. The entire telescopic rod 500 is pushed out, thereby adjusting the height of the mounting box 100.

[0063] like Figure 3 As shown, a miniature drive motor 108 is installed inside the mounting box 100. A friction wheel is installed on the output shaft of the miniature drive motor 108, and the friction wheel is in contact with the outside of the rotating plate 104.

[0064] Specifically, during use, the micro drive motor 108 can be activated to drive the friction wheel to rotate. The rotation of the friction wheel will drive the rotating plate 104 to rotate as a whole. When the rotating plate 104 rotates as a whole, it will drive the outer peripheral bearing bladder 102 and the central bearing bladder 101 to rotate. Thus, the shooting angle of multiple cameras 103 will be changed again while rotating continuously, thereby further increasing the shooting range of the camera 103.

[0065] The technical solutions in the above-described embodiments of this application have at least the following technical effects or advantages: Compared with the prior art, in this embodiment, multiple shots in the same direction can be achieved through the central support capsule 101 and multiple peripheral support capsules 102, and multiple video streams can be output. At the same time, by injecting gas into the peripheral support capsules 102 or the central support capsule 101, the multiple peripheral support capsules 102 or the central support capsule 101 can expand and extend to different degrees, thereby driving multiple cameras 103 to move closer to or further away from the human face in terms of physical distance, and to collect human face data at different distances.

[0066] Example 2: Considering that the human face has a curvature during data acquisition, and that facial muscles undergo different changes when the body makes expressions or micro-expressions, with the jawbone and muscles changing accordingly, current facial data acquisition is conducted from the front. Acquiring side facial data may require staff to use other camera positions, and manual recalibration of the facial contours is necessary during data integration, which is cumbersome. To address these technical problems, this application proposes the following technical solution:

[0067] like Figures 2-7 As shown, each of the multiple peripheral support bladders 102 has an internally integrated internal thrust air bladder 200. The external of the internal thrust air bladder 200 is connected to a first electrically controlled metering valve 201. The outlet of the first electrically controlled metering valve 201 is connected to the outlet of the second electrically controlled metering valve 202. A second electromagnet plate 204 is fixedly connected inside the central support bladder 101, and a first electromagnet plate 203 is fixedly connected inside the peripheral support bladders 102.

[0068] Specifically, during use, when it is necessary to capture the side of a human face, the first electronically controlled metering valve 201 can be opened to deliver gas from the outer peripheral support bladder 102 to the inner push air bladder 200. As the gas continuously enters the inner push air bladder 200, the inner push air bladder 200 will expand. When the inner push air bladder 200 expands, the first electromagnet 203 and the second electromagnet 204 are activated simultaneously. When the first electromagnet 203 and the second electromagnet 204 are activated at the same time, they will generate a magnetic force that attracts each other. When the first electromagnet 203 and the second electromagnet 204 attract each other, they can attract and limit the bottom of the outer peripheral support bladder 102 near the central support bladder 101. Furthermore, as the inner push air bladder 200 continues to expand, it can push the mounting plate 110 and the camera 103 to tilt. When the camera 103 tilts, the side of the human face can be captured.

[0069] like Figure 7 As shown, in Figure 6 The image shows the tilted state of the outer peripheral support bladder 102 at the vertical position. During the acquisition of human facial data, tilting the camera vertically allows for capturing changes in facial details, such as changes in eyebrows and corners of the eyes when a person smiles. Tilting the camera vertically allows for the capture of more detailed information. Furthermore, the expansion of the inner push airbag 200 pushes the outer peripheral support bladder 102. The first electronically controlled metering valve 201 measures the amount of gas entering the inner push airbag 200, controlling the degree of expansion. The inner push airbag 200 expands and pushes the outer peripheral support bladder 102. When the front end of the support bladder 102 tilts, the outer periphery of the support bladder 102 will undergo secondary extension. During this secondary extension and deformation, multiple first strain gauges 109 located outside and inside the outer periphery of the support bladder 102 will extend along with the deformation of the outer periphery of the support bladder 102. As these first strain gauges 109 deform and extend, their resistance values ​​will change. By collecting the resistance changes of the first strain gauges 109 and measuring the amount of gas entering the inner push airbag 200 using the first electrically controlled metering valve 201, the overall tilt angle of the outer periphery of the support bladder 102 can be calculated. The calculation formula is as follows: ,in, The tilt angle of the outer supporting bladder 102. The original length (mm) of the outer supporting bladder (102) is given. The initial resistance (Ω) of the first strain gauge 109 is given. The airbag deformation coefficient is mm³ / Ω. The gas volume (cm³) entering the internal thrust airbag 200 This represents the change in resistance of the first strain gauge 109;

[0070] The formula relating gas volume to capsule extension is as follows: ,in, For deformable variables, This is the proportionality coefficient. The driving quantity is the input variable;

[0071] After obtaining the above relational variables, substitute them into the geometric tilt model. The specific formula of the geometric tilt model is: ,in, The peripheral support bladder is tilted at an angle of 102. cosine value, This refers to the extension of the anterior end of the cyst.

[0072] Furthermore, strain gauge 205 is used to detect the degree of expansion of the central support capsule 101. The central support capsule (101) only expands and extends back and forth, and its calculation formula is as follows: ;

[0073] in, The extension of the central supporting sac 101, The initial length of the central supporting capsule (101) This is the resistance conversion data for the second strain gauge.

[0074] Furthermore, in S4, a micro-expression-based hybrid shape model is reconstructed based on a dense 3D point cloud set, incorporating multiple capsule tilt angle parameters to restructure the model. The calculation formula is as follows:

[0075] ;

[0076] As a baseline constant, a neutral face mesh in an expressionless state;

[0077] Output variables for the target, and finally generate a 3D face model;

[0078] For linear combination terms, there are X basic table cases of basis transformation;

[0079] Predefined base case variants cover global muscle group movements (such as macroscopic expressions like laughter, anger, and surprise).

[0080] The deformation weighting coefficient, ranging from [0, 1], is a continuous value predicted in real time from the video by a neural network. Its function is to control the intensity and blending degree of the basic facial expression.

[0081] For linear combination terms, there are e micro-expression compensation variables;

[0082] For the newly added microsurface variant, describe localized subtle muscle movements (such as crow's feet, nasal flaring, and unilateral twitching of the corner of the mouth).

[0083] The micro-deformation weighting coefficient is used to enhance realism and solve the "rubber face" problem of traditional solutions.

[0084] for example: Deepening of the frown lines between the eyebrows (sadness / focus) : A trembling nasolabial fold on one side (disdain).

[0085] In S5, the specific calculation formula for multi-view error is as follows:

[0086] ;

[0087] Original measurement distance: the straight-line distance from camera 103 to the face;

[0088] Camera 103 tilt angle: The angle between the optical axis of camera 103 and the normal to the face;

[0089] Tilt compensation coefficient: The distance error caused by a unit tilt angle;

[0090] Deformation compensation coefficient: per unit relative deformation The amount of distance error caused;

[0091] Compensation is made based on the error value.

[0092] In S6, the specific calculation formula for the hybrid shape transfer algorithm is as follows:

[0093] ;

[0094] The final driving parameters for the target role;

[0095] The transformation matrix (n×m dimensional) represents a linear transformation that maps the m facial expressions of the source character to the n drivers of the target character.

[0096] Mixed shape field function (spatial location related), based on facial region Dynamically adjust the blending weights (e.g., prioritize mapping eye movements to areas similar to the target character);

[0097] The inverse of the source character rotation matrix, eliminating the influence of the source character's current head rotation (such as coordinate system correction when the actor turns their head).

[0098] The target character rotation matrix adapts the driving data to the orientation of the target character's bones (e.g., coordinate system alignment when the monster character's head is tilted 30°).

[0099] The technical solutions in the above-described embodiments of this application have at least the following technical effects or advantages: Compared with Embodiment 1, in this embodiment, by continuously injecting gas into the interior of the airbag 200, the airbag 200 expands and pushes the outer supporting bag 102 to tilt, thereby causing the camera 103 to tilt. When the multiple cameras 103 tilt, they can capture facial data from the side of the human face. The side data is captured based on the same face, reducing the need for manual splicing and adjustment in the case of multiple cameras.

[0100] Example 3: Considering that during use, the camera 103 located outside the central support bladder 101 faces the human face, and the outer support bladder 102, pushed by the inner airbag 200, may obstruct the light from the camera 103 located outside the central support bladder 101 when tilting to take tilted pictures of the human body from all sides, and that the lighting during the shooting process is particularly important and often requires supplemental lighting, this application proposes the following technical solution to address the above technical problems:

[0101] like Figures 2-8 As shown, the outer surface of the inner thrust airbag 200 is integrally formed with an outwardly expanding reflective bladder 300, which penetrates the outer wall of the outer peripheral support bladder 102. The air inlet of the outwardly expanding reflective bladder 300 is connected to a third metering electronic control valve 301, and the air inlet of the third metering electronic control valve 301 is connected to the inner thrust airbag 200. The outer surface of the outwardly expanding reflective bladder 300 is integrally formed with a reflective layer. The outer surface of the outwardly expanding reflective bladder 300, the central support bladder 101, the outer peripheral support bladder 102, and the outer surface of the inner thrust airbag 200 are all connected to a pressure relief metering valve 302.

[0102] Specifically, during use, when supplemental lighting is needed in one direction, the operator can open the third metering electronic control valve 301 on the outside of the outward-expanding reflector 300 in the opposite direction. With the third metering electronic control valve 301 open, gas can be continuously supplied to the inside of the outward-expanding reflector 300. When the gas is continuously supplied to the inside of the outward-expanding reflector 300, the outward-expanding reflector 300 will expand. When the outward-expanding reflector 300 expands, the reflective layer on the outside of the outward-expanding reflector 300 can reflect the light source projected in the other direction, supplementing the light source to the camera position that needs the light source. In addition, when gas is continuously injected into the airbag 200, the gas pushes the outer supporting bag 102 to expand and tilt, thereby causing the outward-expanding reflector 300 to tilt, further adjusting the supplemental position of the light source.

[0103] Furthermore, during use, when the outward-expanding reflective bladder 300 is tilted, the degree of tilt is the same as the expansion amount of the inner pushing airbag 200 and the tilt angle generated by pushing the outer peripheral bearing bladder 102 and the mounting plate 110. The outer part of the outward-expanding reflective bladder 300 is connected to the mounting plate 110, and when the mounting plate 110 tilts, it can drive the outward-expanding reflective bladder 300 to tilt together.

[0104] Furthermore, when it is not necessary to use components such as the outward-expanding reflector 300 or the inward-pushing airbag 200 for expansion or tilting, the pressure relief metering valve 302 connected to each component can be opened. When the pressure relief metering valve 302 is open, the gas in each component can be discharged to the outside, thereby allowing gas to be discharged between the components connected to the pressure relief metering valve 302. When the gas is discharged, the components connected to the pressure relief metering valve 302 can return to their original positions.

[0105] Considering that during use, multiple cameras 103 require not only supplemental lighting but also light-blocking, and the outer reflective bladder 300 reflects light in real time, while expelling the gas from the outer reflective bladder 300 could eliminate reflection, it would also eliminate the light-blocking effect. Therefore, the following technical solution is proposed:

[0106] like Figure 9 As shown, an electrochromic film 400 is attached to the outside of the reflective layer of the outward-expanding reflective bag 300. An electric conductor is connected to the outside of the electrochromic film 400 and is electrically connected to an external power source.

[0107] Specifically, during use, when light blocking is required, the electrochromic film 400 can be powered on simply by using an external power switch. When the electrochromic film 400 is powered on, it will gradually change color. When the electrochromic film 400 changes color, it can eliminate the reflection of the light source, thereby achieving the light blocking effect. Furthermore, it eliminates the need for frequent inflation and deflation of the external reflective bladder 300, thus improving the overall efficiency of use.

[0108] The technical solutions in the above-described embodiments of this application have at least the following technical effects or advantages: Compared with Embodiment 2, in this embodiment, by continuously supplying gas into the outer reflective bladder 300, the outer reflective bladder 300 can expand. When the outer reflective bladder 300 expands, it can work with the reflective layer to reflect light. In addition, the inner pushing airbag 200 pushes the outer peripheral support bladder 102 and the mounting plate 110 to tilt together, thereby tilting the outer reflective bladder 300. In this tilted state, it can provide supplementary lighting for cameras 103 in different directions. Furthermore, when needed, the reflective layer can be blocked by changing the color state of the electrochromic film 400 to form a light-blocking effect.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A facial motion capture method based on deep learning, characterized in that, Includes the following steps: S1. Set up a dynamic face capture device, collect data from all sides of the face through the dynamic face capture device, and construct a multi-faceted face dataset; S2. Integrates a dynamic face capture device to simultaneously output 3-5 video streams and RGB and depth information; S3. Use a multi-view stereo matching algorithm to generate dense 3D point clouds; S4. Reconstruct the micro-expression-based hybrid shape model based on dense 3D point cloud sets; S5. Generate a multi-view fusion facial action recognition network through a hybrid shape model and calculate the multi-view error; S6. By using a facial motion recognition network and calculating the numerical value of multi-view error, the skeletal topology of different virtual characters in the video is obtained through the deformation transfer algorithm. The dynamic face capture device in S1 includes a mounting box (100). A rotating plate (104) is rotatably connected inside the mounting box (100). A central support capsule (101) and multiple peripheral support capsules (102) are fixedly connected to the outside of the rotating plate (104). The multiple peripheral support capsules (102) surround the central support capsule (101). A mounting plate (110) is fixedly connected to the side of the central support capsule (101) and the peripheral support capsules (102) away from the rotating plate (104). A camera (103) is installed inside the mounting plate (110). An air pump (105) is installed inside the mounting box (100). The air pump (105) outputs air... The port is connected to a delivery pipe (106), and a rotary joint (107) is installed at the end of the delivery pipe (106) away from the air pump (105). The rotating air outlet of the rotary joint (107) penetrates the outer wall of the rotating plate (104) and is connected to the central support bladder (101). The outside of the central support bladder (101) is connected to a plurality of second electrically controlled metering valves (202), and the plurality of second electrically controlled metering valves (202) are respectively connected to a plurality of peripheral support bladders (102). A micro drive motor (108) is installed inside the mounting box (100), and a friction wheel is installed on the output shaft of the micro drive motor (108). The friction wheel is in contact with the outside of the rotating plate (104). Each of the multiple peripheral support bladders (102) has an internally formed internal pushing air bladder (200) integrated inside. The external of the internal pushing air bladder (200) is connected to a first electrically controlled metering valve (201). The outlet of the first electrically controlled metering valve (201) is connected to the outlet of the second electrically controlled metering valve (202). A second electromagnet plate (204) is fixedly connected inside the central support bladder (101), and a first electromagnet plate (203) is fixedly connected inside the peripheral support bladders (102). The central support capsule (101) and the peripheral support capsule (102) are placed in front of the human face. Multiple cameras (103) capture images of the human face and output multiple video streams. The central support capsule (101) and the peripheral support capsule (102) are expanded by activating the air pump (105), which causes the cameras (103) to extend forward, thereby adjusting the shooting distance and collecting human face data at different distances. By delivering the gas inside the outer supporting bladder (102) to the inner pushing airbag (200), the first electromagnet (203) and the second electromagnet (204) are activated at the same time as the inner pushing airbag (200) expands. When the first electromagnet (203) and the second electromagnet (204) attract each other, the inner pushing airbag (200) pushes the mounting plate (110) and the camera (103) to tilt, so as to capture the side view of the human face.

2. The facial motion capture method based on deep learning according to claim 1, characterized in that: The outer surface of the inner propulsion airbag (200) is integrally formed with an outwardly expanding reflective bladder (300), which penetrates the outer wall of the outer peripheral support bladder (102). The air inlet of the outwardly expanding reflective bladder (300) is connected to a third metering electronic control valve (301), and the air inlet of the third metering electronic control valve (301) is connected to the inner propulsion airbag (200). The outer surface of the outwardly expanding reflective bladder (300) is integrally formed with a reflective layer. The outer surface of the outwardly expanding reflective bladder (300), the central support bladder (101), the outer peripheral support bladder (102), and the inner propulsion airbag (200) are all connected to a pressure relief metering valve (302).

3. The facial motion capture method based on deep learning according to claim 2, characterized in that: An electrochromic film (400) is attached to the outside of the reflective layer of the outward-expanding reflective bag (300), and an electric conductor is connected to the outside of the electrochromic film (400), which is electrically connected to an external power source.

4. The facial motion capture method based on deep learning according to claim 3, characterized in that: The bottom of the mounting box (100) is fixedly connected to a telescopic rod (500), and the bottom of the telescopic rod (500) is fixedly connected to a base (501).

5. The facial motion capture method based on deep learning according to claim 4, characterized in that: The external of the delivery pipe (106) is connected to an external delivery valve (800). The end of the external delivery valve (800) away from the delivery pipe (106) passes through the outer wall of the rotating plate (104) and the telescopic rod (500) and is connected to an extension pipe (801). An internally pushed multi-stage moving bladder (802) is fixedly connected inside the telescopic rod (500). The air outlet of the extension pipe (801) is connected to the internally pushed multi-stage moving bladder (802).

6. The facial motion capture method based on deep learning according to claim 5, characterized in that: The base (501) has a sliding groove inside, and a T-shaped slider (600) is slidably connected inside the sliding groove. A fastening band (601) is fixedly connected to the bottom of the T-shaped slider (600).

7. The facial motion capture method based on deep learning according to claim 6, characterized in that: A suction cup (700) is fixedly connected to the bottom of the base (501).

8. The facial motion capture method based on deep learning according to claim 7, characterized in that: Multiple second strain gauges (205) are fixedly connected to the outside of the central support bladder (101), and multiple first strain gauges (109) are fixedly connected to the outside of both the inner pushing airbag (200) and the outer peripheral support bladder (102). The first strain gauges (109) and the second strain gauges (205) are used to generate resistance changes when the central support bladder (101) and the outer peripheral support bladder (102) expand and extend. The angular position of the camera (103) is calculated by the resistance change and the expansion amount. The calculation formula is as follows: ; in, The tilt angle of the outer supporting bladder (102); The original length of the peripheral support capsule (102); The initial resistance of the first strain gauge (109); This is the airbag deformation coefficient; The volume of gas entering the internal thrust airbag (200); The change in resistance of the first strain gauge (109); The formula relating gas volume to capsule extension is as follows: ,in, For deformable variables, This is the proportionality coefficient. The driving quantity is the input variable; After obtaining the above relational variables, substitute them into the geometric tilt model. The specific formula of the geometric tilt model is: ; in, The tilt angle of the peripheral bearing bladder (102) The cosine value; This refers to the extension of the anterior end of the cyst.

Images