Facial motion capture method based on deep learning

Abstract

The invention discloses a facial motion capture method based on deep learning, and belongs to the technical field of facial capture, and the method comprises the following steps: S1, erecting a dynamic face capture device, collecting the data of each face through the dynamic face capture device, and constructing a face multi-face data set; s2, integrating 3-5 paths of video streams synchronously output by the dynamic face capturing device, RGB information and depth information; s3, generating a dense 3D point cloud by using a multi-view stereo matching algorithm; a multi-view error is calculated, gas is continuously injected into an inner pushing air bag, so that a peripheral bearing bag body can be pushed to incline under the condition that the inner pushing air bag expands, the cameras are driven to incline, and the multiple cameras can capture face data of the side face of the human face under the condition that the cameras incline. Side surface data acquisition and capture are integrally carried out based on the same face, and the phenomenon that multiple times of manual splicing adjustment are needed under the multi-camera condition is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face capture, in particular to a face motion capture method based on deep learning. BACKGROUND

[0002] In the field of film and television animation production, generating character animation by capturing real face motion is a key technical link to improve the realism and expressiveness of works. Traditional face motion capture technology mainly relies on optical markers or mechanical devices, records the motion trajectory of key points on the face through a multi-angle camera array or sensor network, and then drives the expression changes of digital characters. The breakthrough of deep learning technology brings revolutionary changes to face motion capture. Based on the end-to-end solution of convolutional neural network (CNN) and recurrent neural network (RNN), real-time face video stream is collected through infrared camera or ordinary RGB camera, and pre-trained model is used to automatically extract key feature points and generate three-dimensional expression parameters, which solves the problems in the traditional capture process.

[0003] In the existing patent, it is pointed out that the current mainstream methods can be divided into two-dimensional data-based and three-dimensional data-based. The former uses optical lens to understand human facial expressions and actions through algorithms, such as Faceware's helmet-style single-camera face motion capture system. The advantages of this method are low cost, easy to obtain, and convenient to use, and the disadvantages are lower capture accuracy compared with other methods; the latter obtains two-dimensional data through optical lens, and obtains depth information through additional means or devices, such as multi-camera and structured light. This method has fast processing speed and high accuracy, but requires additional depth acquisition devices. It solves the problem of operation efficiency of traditional capture by outputting mixed shape coefficients and applying the coefficients to virtual images for real-time rendering of expressions. However, the overall method still uses a single camera to capture images, and the overall method only uses algorithms to complete the missing images. However, in the field of film and television animation, especially in 3D films, the micro-expression of human face will still appear distortion phenomenon when completing, which affects the output content, and still needs a lot of manual calibration in the later stage. Moreover, the existing multi-camera still captures human face from the front based on the current angle, and the capture of muscle conditions on the side of human face is still poor. Therefore, a face motion capture method based on deep learning is proposed. SUMMARY

[0004] The present application aims to provide a face motion capture method based on deep learning to solve the problems in the background technology.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: a face motion capture method based on deep learning, comprising the following steps: S1, erect a dynamic face capture device, collect face data of each face through the dynamic face capture device, and construct a multi-face data set of the face; S2, integrate the dynamic face capture device to synchronously output a video stream and RGB information and depth information; S3, generate a dense 3D point cloud using a multi-view stereo matching algorithm; S4, reconstruct a micro-expression basis hybrid shape model based on the dense D point cloud set; S5, generate a multi-view fusion facial action recognition network through the hybrid shape model, and calculate a multi-view error; S6, pass the numerical value of the facial action recognition network and the calculated multi-view error through a deformation transfer algorithm to video different virtual image bone topologies.

[0006] Preferably, the dynamic face capture device in S1 comprises a mounting box, a rotating plate is rotatably connected inside the mounting box, a central bearing capsule and a plurality of peripheral bearing capsules are fixedly connected outside the rotating plate, the plurality of peripheral bearing capsules surround the central bearing capsule, mounting plates are fixedly connected to the sides of the central bearing capsule and the peripheral bearing capsules away from the rotating plate, cameras are installed inside the mounting plates, a gas pump is installed inside the mounting box, a delivery pipe is communicated with the gas outlet of the gas pump, a rotary joint is installed at the end of the delivery pipe away from the gas pump, the rotary gas outlet of the rotary joint penetrates the outer wall of the rotating plate and is communicated with the central bearing capsule, a plurality of second electric control metering valves are communicated with the outside of the central bearing capsule, the plurality of second electric control metering valves are respectively communicated with the plurality of peripheral bearing capsules, a micro drive motor is installed inside the mounting box, a friction wheel is installed on the output shaft of the micro drive motor, and the friction wheel is attached to the outside of the rotating plate.

[0007] Preferably, an inner push air bag is integrally formed inside each of the plurality of peripheral bearing capsules, a first electric control metering valve is communicated with the outside of the inner push air bag, the first electric control metering valve is communicated with the gas outlets of the second electric control metering valves, a second electromagnet plate is fixedly connected inside the central bearing capsule, and a first electromagnet plate is fixedly connected inside the peripheral bearing capsule.

[0008] Preferably, an outer expansion light reflection capsule is integrally formed outside the inner push air bag, the outer expansion light reflection capsule penetrates the outer wall of the peripheral bearing capsule, a third metering electric control valve is communicated with the gas inlet of the outer expansion light reflection capsule, the gas inlet of the third metering electric control valve is communicated with the inner push air bag, a light reflection layer is integrally formed outside the outer expansion light reflection capsule, and a pressure relief metering valve is communicated with the outside of the outer expansion light reflection capsule, the central bearing capsule, the peripheral bearing capsule, and the inner push air bag.

[0009] Preferably, the outer surface of the reflective layer of the reflective light-emitting capsule is attached with an electrochromic film, and the electrochromic film is externally connected with a power supply wire which is electrically connected with an external power source.

[0010] Preferably, the bottom of the mounting box is fixedly connected with an extension rod, and the bottom of the extension rod is fixedly connected with a base.

[0011] Preferably, the outer surface of the delivery pipe is connected with an external delivery valve, one end of the external delivery valve is penetrated through the outer wall of the rotating plate and the extension rod and is connected with an extension pipe, the inner surface of the extension rod is fixedly connected with an inner push multi-stage dynamic capsule, and the gas outlet of the extension pipe is connected with the inner push multi-stage dynamic capsule.

[0012] Preferably, the inner surface of the base is provided with a sliding groove, and the sliding groove is slidably connected with a T-shaped sliding block, and the bottom of the T-shaped sliding block is fixedly connected with a fastening belt.

[0013] Preferably, the bottom of the base is fixedly connected with a suction cup.

[0014] Preferably, the outer surface of the central bearing capsule is fixedly connected with a plurality of second strain gauges, the outer surface of the inner push air bag and the outer peripheral bearing capsule is fixedly connected with a plurality of first strain gauges, the first strain gauges and the second strain gauges are used to generate resistance changes when the central bearing capsule and the outer peripheral bearing capsule are inflated and extended, the angle position of the camera is calculated through the resistance changes and the inflation amount, and the calculation formula is: ; wherein, is the inclination angle of the outer peripheral bearing capsule; is the original length of the outer peripheral bearing capsule; is the initial resistance of the first strain gauge; is the deformation coefficient of the air bag; is the gas volume entering the inner push air bag; is the resistance change amount of the first strain gauge; wherein, the relationship formula between the gas volume and the capsule extension amount is: wherein, is the deformation amount, is the proportional coefficient, is the driving amount, i.e., the input variable; After the above relationship variables are obtained, the geometric inclination model is substituted, and the specific formula of the geometric inclination model is: ; wherein, is the inclination angle of the outer peripheral bearing capsule cosine value of the angle between the first vector and the second vector; The extension amount of the front end of the capsule.

[0015] Compared with the prior art, the present application has the following advantages: In the present application, multiple shooting in the same direction can be realized by the central bearing capsule and the plurality of peripheral bearing capsules, and multiple video stream outputs can be realized, and at the same time, by injecting gas into the inside of the peripheral bearing capsule or the central bearing capsule, the plurality of peripheral bearing capsules or the central bearing capsule can be expanded and extended to different degrees, thereby driving the plurality of cameras to approach or move away from the human face in physical distance, and collecting human face data at different distances.

[0016] In the present application, by continuously injecting gas into the inside of the air bag, the inner pushing air bag can be expanded to push the peripheral bearing capsule to tilt, thereby driving the camera to tilt, and the plurality of cameras can capture face data on the side of the human face under the condition of tilting, and the side data of the same face is collected and captured, reducing the phenomenon of manual multiple splicing and adjustment under the condition of multiple positions. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The flow structure diagram in the embodiment of the present application; Figure 2 The structure diagram of the dynamic face capturing device in the embodiment of the present application; Figure 3 The air pump structure diagram of the dynamic face capturing device in the embodiment of the present application; Figure 4 The internal structure diagram of the peripheral bearing capsule of the dynamic face capturing device in the embodiment of the present application; Figure 5 The cross-sectional structure diagram of the central bearing capsule of the dynamic face capturing device in the embodiment of the present application; Figure 6 The three-dimensional structure diagram of the central bearing capsule of the dynamic face capturing device in the embodiment of the present application; Figure 7 The structure diagram of the tilting state of the peripheral bearing capsule in the dynamic face capturing device in the embodiment of the present application; Figure 8 The structure diagram of the expansion state of the outwardly expanding reflective capsule in the dynamic face capturing device in the embodiment of the present application; Figure 9 The structure diagram of the electrochromic film in the dynamic face capturing device in the embodiment of the present application; Figure 10 The cross-sectional structure diagram of the telescopic rod in the dynamic face capturing device in the embodiment of the present application.

[0018] In the figure: 100, installation box; 101, central bearing capsule; 102, peripheral bearing capsule; 103, camera; 104, rotating plate; 105, air pump; 106, conveying pipe; 107, rotary joint; 108, micro drive motor; 109, first strain gauge; 110, mounting plate; 200, inner push air bag; 201, first electric control metering valve; 202, second electric control metering valve; 203, first electromagnet sheet; 204, second electromagnet sheet; 205, second strain gauge; 300, outer expansion light reflecting capsule; 301, third metering electric control valve; 302, pressure relief metering valve; 400, electrochromic film; 500, telescopic rod; 501, base; 600, T-shaped slider; 601, fastening belt; 700, suction cup; 800, external conveying valve; 801, extension pipe; 802, inner push multi-stage dynamic capsule. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0020] Embodiment one, as shown in the present application, a face action capture method based on deep learning, comprising the following steps: Figure 1 S1, erecting a dynamic face capture device, collecting each face data of the face through the dynamic face capture device, and constructing a multi-face data set of the face; S2, integrating the dynamic face capture device to synchronously output 3-5 video streams and RGB information and depth information; S3, using a multi-view stereo matching algorithm to generate a dense 3D point cloud; S4, reconstructing a micro-expression base hybrid shape model based on the dense 3D point cloud set; S5, generating a multi-view fusion face action recognition network through the hybrid shape model, and calculating a multi-view error; S6, passing the numerical value of the face action recognition network and the calculated multi-view error through a deformation migration algorithm to topologically expand the skeleton of different virtual images. As shown in the present application, a face action capture method based on deep learning, comprising the following steps:

[0021] S1, erecting a dynamic face capture device, collecting each face data of the face through the dynamic face capture device, and constructing a multi-face data set of the face; Figures 1-7As shown, the dynamic face capturing device in S1 includes a mounting box 100, a rotating plate 104 is rotatably connected inside the mounting box 100, a central bearing capsule 101 and a plurality of peripheral bearing capsules 102 are fixedly connected outside the rotating plate 104 respectively, the plurality of peripheral bearing capsules 102 surround the central bearing capsule 101, a mounting plate 110 is fixedly connected to the side of the central bearing capsule 101 and the peripheral bearing capsules 102 away from the rotating plate 104, a camera 103 is installed inside the mounting plate 110, a gas pump 105 is installed inside the mounting box 100, a delivery pipe 106 is communicated with the gas outlet of the gas pump 105, a rotary joint 107 is installed at one end of the delivery pipe 106 away from the gas pump 105, the rotary outlet of the rotary joint 107 penetrates the outer wall of the rotating plate 104 and is communicated with the central bearing capsule 101, a plurality of second electric control metering valves 202 are communicated with the outside of the central bearing capsule 101, and the plurality of second electric control metering valves 202 are respectively communicated with the plurality of peripheral bearing capsules 102.

[0022] Specifically, in use, when data collection of the face of the film and television staff is needed, the central bearing capsule 101 and the peripheral bearing capsule 102 can be placed in front of the human face, after placement, the camera 103 is started to realize shooting of the human face by the plurality of cameras 103, outputting a plurality of video streams, and in use, the gas pump 105 can also be started to continuously inject gas into the inside of the delivery pipe 106, when the gas is continuously injected into the inside of the delivery pipe 106, the delivery pipe 106 will deliver the gas to the inside of the rotary joint 107, the gas will be delivered to the inside of the central bearing capsule 101 through the rotary joint 107 when delivered to the inside of the rotary joint 107, the central bearing capsule 101 will expand when the gas enters the inside of the central bearing capsule 101, the central bearing capsule 101 can drive the camera 103 to stretch forward to adjust the shooting distance.

[0023] Further, the plurality of second electric control metering valves 202 can also be started to inject the gas in the central bearing capsule 101 into the inside of the peripheral bearing capsule 102, when the gas is injected into the inside of the peripheral bearing capsule 102, the plurality of peripheral bearing capsules 102 will also expand, the expansion of the peripheral bearing capsule 102 will drive the mounting plate 110 and the camera 103 to stretch forward when the peripheral bearing capsule 102 expands, so that the whole is close to or away from the human face position during shooting to realize multiple shooting of near focus or far focus, reducing the problem that the multiple cameras may produce the phenomenon of blurring.

[0024] As shown in the figure, Figures 1-2 the bottom of the mounting box 100 is fixedly connected with an extension rod 500, and the bottom of the extension rod 500 is fixedly connected with a base 501.

[0025] Specifically, during use, when it is necessary to align the position of the mounting box 100 with the face of the person being photographed, the height of the mounting box 100 can be adjusted by adjusting the telescopic length of the telescopic rod 500.

[0026] like Figures 2-3 As shown, a sliding groove is provided inside the base 501, and a T-shaped slider 600 is slidably connected inside the sliding groove. A fastening band 601 is fixedly connected to the bottom of the T-shaped slider 600, and a suction cup 700 is fixedly connected to the bottom of the base 501.

[0027] Specifically, when performing wire stunts and other similar projects that require capturing the faces of the person being filmed during the wire stunt process, the T-shaped slider 600 can be inserted into the sliding groove, and the T-shaped slider 600 can be fixed to the designated wire crossbar by the fastening strap 601. This fixes and limits the position of the telescopic rod 500 and the mounting box 100. Furthermore, the suction cup 700 ensures the stability between the base 501 and the placement surface when the base 501 is placed in the designated position.

[0028] like Figure 10 As shown, an external delivery valve 800 is connected to the outside of the delivery pipe 106. The end of the external delivery valve 800 away from the delivery pipe 106 passes through the outer wall of the rotating plate 104 and the telescopic rod 500 and is connected to an extension pipe 801. An internally pushed multi-stage moving bladder 802 is fixedly connected inside the telescopic rod 500. The air outlet of the extension pipe 801 is connected to the internally pushed multi-stage moving bladder 802.

[0029] Specifically, during use, when it is necessary to actively adjust the telescopic distance of the telescopic rod 500, gas can be continuously injected into the delivery pipe 106 by starting the air pump 105 and closing the rotary joint 107. With the rotary joint 107 closed, the gas will be continuously delivered to the external delivery valve 800. When the gas is delivered to the external delivery valve 800, it will be delivered to the internal multi-stage moving bladder 802 through the extension pipe 801. When gas is continuously entering the internal multi-stage moving bladder 802, the internal multi-stage moving bladder 802 will expand. When the internal multi-stage moving bladder 802 expands, it will push the telescopic rod 500, thereby causing the multi-section telescopic sections of the telescopic rod 500 to extend. The entire telescopic rod 500 is pushed out, thereby adjusting the height of the mounting box 100.

[0030] like Figure 3 As shown, a miniature drive motor 108 is installed inside the mounting box 100. A friction wheel is installed on the output shaft of the miniature drive motor 108, and the friction wheel is in contact with the outside of the rotating plate 104.

[0031] Specifically, during use, the friction wheel can be driven to rotate by starting the micro drive motor 108, and the rotation of the friction wheel can drive the rotating plate 104 to rotate as a whole, and the rotation of the rotating plate 104 as a whole can drive the outer peripheral bearing capsule 102 and the central bearing capsule 101 to rotate, so that the shooting angles of the plurality of cameras 103 are changed again under continuous rotation, and the shooting range of the cameras 103 is further increased.

[0032] The technical solutions in the embodiments of the present application have at least the following technical effects or advantages: compared with the prior art, in the embodiments, multiple shooting in the same direction can be achieved by the central bearing capsule 101 and the plurality of outer peripheral bearing capsules 102, multiple video stream outputs are achieved, and the plurality of outer peripheral bearing capsules 102 or the central bearing capsule 101 can be inflated by injecting gas into the inside of the plurality of outer peripheral bearing capsules 102 or the central bearing capsule 101, so that the plurality of cameras 103 are driven to move closer to or farther away from the human face in physical distance, and human face data is collected at different distances.

[0033] Embodiment two, considering that the human face has a curvature during data collection of the human face, the facial muscles change differently when the human face produces an expression or a micro-expression, the jawbone and muscles change with the change of the expression, and the current face data collection is performed in the front direction, the collection of the side face may need to rely on the staff to use other positions to shoot and collect, and manual re-calibration of the connection position of the human face is needed in integration, which is troublesome, and the above technical problems are solved by the technical solutions proposed in the present application, specifically: As shown in Figures 2-7 The inside of each of the plurality of outer peripheral bearing capsules 102 is integrally formed with an inner push air bag 200, the outer portion of the inner push air bag 200 is communicated with a first electric control metering valve 201, the first electric control metering valve 201 is communicated with the gas outlet of a second electric control metering valve 202, the inside of the central bearing capsule 101 is fixedly connected with a second electromagnet sheet 204, and the inside of the outer peripheral bearing capsule 102 is fixedly connected with a first electromagnet sheet 203.

[0034] Specifically, during use, when it is necessary to capture the side of a human face, the first electronically controlled metering valve 201 can be opened to deliver gas from the outer peripheral support bladder 102 to the inner push air bladder 200. As the gas continuously enters the inner push air bladder 200, the inner push air bladder 200 will expand. When the inner push air bladder 200 expands, the first electromagnet 203 and the second electromagnet 204 are activated simultaneously. When the first electromagnet 203 and the second electromagnet 204 are activated at the same time, they will generate a magnetic force that attracts each other. When the first electromagnet 203 and the second electromagnet 204 attract each other, they can attract and limit the bottom of the outer peripheral support bladder 102 near the central support bladder 101. Furthermore, as the inner push air bladder 200 continues to expand, it can push the mounting plate 110 and the camera 103 to tilt. When the camera 103 tilts, the side of the human face can be captured.

[0035] like Figure 7 As shown, in Figure 6 The image shows the tilted state of the outer peripheral support bladder 102 at the vertical position. During the acquisition of human facial data, tilting the camera vertically allows for capturing changes in facial details, such as changes in eyebrows and corners of the eyes when a person smiles. Tilting the camera vertically allows for the capture of more detailed information. Furthermore, the expansion of the inner push airbag 200 pushes the outer peripheral support bladder 102. The first electronically controlled metering valve 201 measures the amount of gas entering the inner push airbag 200, controlling the degree of expansion. The inner push airbag 200 expands and pushes the outer peripheral support bladder 102. When the front end of the support bladder 102 tilts, the outer periphery of the support bladder 102 will undergo secondary extension. During this secondary extension and deformation, multiple first strain gauges 109 located outside and inside the outer periphery of the support bladder 102 will extend along with the deformation of the outer periphery of the support bladder 102. As these first strain gauges 109 deform and extend, their resistance values ​​will change. By collecting the resistance changes of the first strain gauges 109 and measuring the amount of gas entering the inner push airbag 200 using the first electrically controlled metering valve 201, the overall tilt angle of the outer periphery of the support bladder 102 can be calculated. The calculation formula is as follows: ,in, The tilt angle of the outer supporting bladder 102. The original length (mm) of the outer supporting bladder (102) is given. The initial resistance (Ω) of the first strain gauge 109 is given. The airbag deformation coefficient is mm³ / Ω. The gas volume (cm³) entering the internal thrust airbag 200 This represents the change in resistance of the first strain gauge 109; The formula relating gas volume to capsule extension is as follows: ,in, For deformable variables, This is the proportionality coefficient. The driving quantity is the input variable; After obtaining the above relational variables, substitute them into the geometric tilt model. The specific formula of the geometric tilt model is: ,in, The peripheral support bladder is tilted at an angle of 102. cosine value, This refers to the extension of the anterior end of the cyst.

[0036] Furthermore, strain gauge 205 is used to detect the degree of expansion of the central support capsule 101. The central support capsule (101) only expands and extends back and forth, and its calculation formula is as follows: ; in, The extension of the central supporting sac 101, The initial length of the central supporting capsule (101) This is the resistance conversion data for the second strain gauge.

[0037] Furthermore, in S4, a micro-expression-based hybrid shape model is reconstructed based on a dense 3D point cloud set, incorporating multiple capsule tilt angle parameters to restructure the model. The calculation formula is as follows: ; As a baseline constant, a neutral face mesh in an expressionless state; Output variables for the target, and finally generate a 3D face model; For linear combination terms, there are X basic table cases of basis transformation; Predefined base case variants cover global muscle group movements (such as macroscopic expressions like laughter, anger, and surprise). The deformation weighting coefficient, ranging from [0, 1], is a continuous value predicted in real time from the video by a neural network. Its function is to control the intensity and blending degree of the basic facial expression. For linear combination terms, there are e micro-expression compensation variables; For the newly added microsurface variant, describe localized subtle muscle movements (such as crow's feet, nasal flaring, and unilateral twitching of the corner of the mouth). Micro-deformation weight coefficient, which enhances the sense of reality and solves the "rubber face" problem of traditional solutions.

[0038] For example: : deepening of the glabella furrow (sadness / concentration), : unilateral nasolabial fold flutter (contempt).

[0039] In S5, the multi-view error is calculated, and the specific calculation formula is: ; Original measurement distance: the straight-line distance from the camera 103 to the face; Camera 103 tilt angle: the angle between the camera 103 optical axis and the face normal; Tilt compensation coefficient: the distance error caused by a unit tilt angle; Deformation compensation coefficient: the distance error caused by a unit relative deformation ; Compensate by error value.

[0040] In S6, the specific calculation formula of the mixed shape migration algorithm is: ; Final driving parameters for the target character; Affine transformation matrix (n x m dimension), which maps the source character m expression bases to the linear conversion relationship of the target character n drivers; Mixed shape field function (related to spatial position), dynamically adjusts the mixing weight according to the face area (eg. Eye movement is preferentially mapped to the target character's similar area); Inverse matrix of the source character rotation matrix, eliminating the influence of the current head rotation of the source character (eg. Coordinate system correction when the actor turns his head); Target character rotation matrix, which adapts the driving data to the orientation of the target character's skeleton (eg. Coordinate system alignment when the monster character's head is tilted 30°).

[0041] The technical solutions in the embodiments of the present application have at least the following technical effects or advantages: compared with the first embodiment, in the present embodiment, the internal injection of gas continuously pushes the inner pushing air bag 200, so that the outer peripheral bearing bag body 102 can be pushed to tilt when the inner pushing air bag 200 expands, thereby driving the camera 103 to tilt, and the plurality of cameras 103 can capture face data on the side of the human face when tilting, and the side data is captured based on the same face, thereby reducing the phenomenon that manual multiple splicing and adjustment are required in the multi-camera case.

[0042] In the third embodiment, considering that the camera 103 located outside the central bearing bag body 101 faces the human face during use, and the outer peripheral bearing bag body 102 pushed by the inner pushing air bag 200 may block the light of the camera 103 located outside the central bearing bag body 101 when tilting to capture the upper, lower, left and right faces of the human body, and the light during the shooting process is also particularly important, and light supplement is often required, to solve the above technical problems, the present application proposes the following technical solutions, in particular: As shown in Figures 2-8 The outer expansion light reflecting bag 300 is integrally formed on the outside of the inner pushing air bag 200, the outer expansion light reflecting bag 300 penetrates the outer wall of the outer peripheral bearing bag body 102, the gas inlet of the outer expansion light reflecting bag 300 is communicated with the third metering electric valve 301, the gas inlet of the third metering electric valve 301 is communicated with the inner pushing air bag 200, the outer expansion light reflecting bag 300 is integrally formed with a light reflecting layer on the outside, and the outside of the outer expansion light reflecting bag 300, the central bearing bag body 101, the outer peripheral bearing bag body 102 and the inner pushing air bag 200 are communicated with the pressure relief metering valve 302.

[0043] Specifically, during use, when light supplement is required in one direction, the staff can open the third metering electric valve 301 outside the outer expansion light reflecting bag 300 in the opposite direction, and the gas can be continuously delivered to the inside of the outer expansion light reflecting bag 300 under the condition that the third metering electric valve 301 is opened, when the gas is continuously delivered to the inside of the outer expansion light reflecting bag 300, the outer expansion light reflecting bag 300 will expand, and the light reflecting layer outside the outer expansion light reflecting bag 300 can reflect the light source projected in the other direction to supply light to the camera position requiring light, and by continuously injecting gas into the inside of the inner pushing air bag 200, the gas pushes the outer peripheral bearing bag body 102 to expand and tilt, thereby driving the outer expansion light reflecting bag 300 to tilt, and further adjusting the light supplement position.

[0044] Further, in use, the outer expansion light reflection bag 300 is inclined, and the degree of inclination is the same as the expansion amount of the inner pushing air bag 200 and the inclination angle generated by pushing the outer peripheral bearing bag body 102 and the mounting plate 110. The outer part of the outer expansion light reflection bag 300 is connected to the mounting plate 110, and the mounting plate 110 can be inclined to drive the outer expansion light reflection bag 300 to incline.

[0045] Further, without the need to use the outer expansion light reflection bag 300 or the inner pushing air bag 200 to expand or push the inclination, the pressure relief metering valve 302 connected to each component can be opened. When the pressure relief metering valve 302 is opened, the gas in each component can be discharged to the outside, so that the gas is discharged between the components connected to the pressure relief metering valve 302, and the components connected to the pressure relief metering valve 302 can be restored to the original position.

[0046] Considering that during use, the shooting of the plurality of cameras 103 not only needs to be lighted, but also needs to be shaded, and the outer expansion light reflection bag 300 is real-time reflection, if the gas in the outer expansion light reflection bag 300 is discharged, although the reflection can be cancelled, but at the same time, shading cannot be performed. Therefore, the following technical solution is proposed: As shown in Figure 9 The outer part of the reflection layer of the outer expansion light reflection bag 300 is attached with an electrochromic film 400, and the electrochromic film 400 is externally connected with a power supply wire, and the power supply wire is electrically connected with an external power supply.

[0047] Specifically, during use, when shading is needed, the electrochromic film 400 can be powered on only by an external power supply switch. When the electrochromic film 400 is powered on, the electrochromic film 400 will gradually change color. When the electrochromic film 400 changes color, the reflection of the light source can be cancelled, thereby achieving the shading effect, and frequent charging and discharging of the outer expansion light reflection bag 300 is not needed, thereby improving the overall use efficiency.

[0048] The technical solution in the embodiment of the present application has at least the following technical effects or advantages: compared with embodiment two, in the present embodiment, the outer expansion light reflection bag 300 can be expanded by continuously supplying gas to the inside of the outer expansion light reflection bag 300. When the outer expansion light reflection bag 300 is expanded, the light can be reflected by the reflection layer, and the inclination of the outer expansion light reflection bag 300 can be driven by the inclination of the outer peripheral bearing bag body 102 and the mounting plate 110 pushed by the inner pushing air bag 200, thereby performing light supplementing on the cameras 103 in different directions in the inclined state. When needed, the reflection layer can be shielded by changing the color state of the electrochromic film 400, thereby forming a shading effect.

[0049] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A facial motion capture method based on deep learning, characterized in that, Includes the following steps: S1. Set up a dynamic face capture device, collect data from all sides of the face through the dynamic face capture device, and construct a multi-faceted face dataset; S2. Integrates a dynamic face capture device to simultaneously output 3-5 video streams and RGB and depth information; S3. Use a multi-view stereo matching algorithm to generate dense 3D point clouds; S4. Reconstruct the micro-expression-based hybrid shape model based on dense 3D point cloud sets; S5. Generate a multi-view fusion facial action recognition network through a hybrid shape model and calculate the multi-view error; S6. By using a facial motion recognition network and calculating the numerical value of multi-view error, the skeletal topology of different virtual characters in the video is obtained through a deformation transfer algorithm.

2. The facial motion capture method based on deep learning according to claim 1, characterized in that: The dynamic face capture device in S1 includes a mounting box (100). A rotating plate (104) is rotatably connected inside the mounting box (100). A central support capsule (101) and multiple peripheral support capsules (102) are fixedly connected to the outside of the rotating plate (104). The multiple peripheral support capsules (102) surround the central support capsule (101). A mounting plate (110) is fixedly connected to the side of the central support capsule (101) and the peripheral support capsules (102) away from the rotating plate (104). A camera (103) is installed inside the mounting plate (110). An air pump (105) is installed inside the mounting box (100). The air pump (105) outputs air... The port is connected to a delivery pipe (106), and a rotary joint (107) is installed at the end of the delivery pipe (106) away from the air pump (105). The rotating air outlet of the rotary joint (107) penetrates the outer wall of the rotating plate (104) and is connected to the central support bladder (101). The outside of the central support bladder (101) is connected to a plurality of second electrically controlled metering valves (202), and the plurality of second electrically controlled metering valves (202) are respectively connected to a plurality of peripheral support bladders (102). A micro drive motor (108) is installed inside the mounting box (100), and a friction wheel is installed on the output shaft of the micro drive motor (108). The friction wheel is in contact with the outside of the rotating plate (104).

3. The facial motion capture method based on deep learning according to claim 2, characterized in that: Each of the multiple peripheral support bladders (102) has an internally formed internal pushing air bladder (200) inside. The external part of the internal pushing air bladder (200) is connected to a first electrically controlled metering valve (201). The outlet of the first electrically controlled metering valve (201) is connected to the outlet of the second electrically controlled metering valve (202). The internal part of the central support bladder (101) is fixedly connected to a second electromagnet (204), and the internal part of the peripheral support bladder (102) is fixedly connected to a first electromagnet (203).

4. The facial motion capture method based on deep learning according to claim 3, characterized in that: The outer surface of the inner propulsion airbag (200) is integrally formed with an outwardly expanding reflective bladder (300), which penetrates the outer wall of the outer peripheral support bladder (102). The air inlet of the outwardly expanding reflective bladder (300) is connected to a third metering electronic control valve (301), and the air inlet of the third metering electronic control valve (301) is connected to the inner propulsion airbag (200). The outer surface of the outwardly expanding reflective bladder (300) is integrally formed with a reflective layer. The outer surface of the outwardly expanding reflective bladder (300), the central support bladder (101), the outer peripheral support bladder (102), and the inner propulsion airbag (200) are all connected to a pressure relief metering valve (302).

5. The facial motion capture method based on deep learning according to claim 4, characterized in that: An electrochromic film (400) is attached to the outside of the reflective layer of the outward-expanding reflective bag (300), and an electric conductor is connected to the outside of the electrochromic film (400), which is electrically connected to an external power source.

6. The facial motion capture method based on deep learning according to claim 5, characterized in that: The bottom of the mounting box (100) is fixedly connected to a telescopic rod (500), and the bottom of the telescopic rod (500) is fixedly connected to a base (501).

7. The facial motion capture method based on deep learning according to claim 6, characterized in that: The external of the delivery pipe (106) is connected to an external delivery valve (800). The end of the external delivery valve (800) away from the delivery pipe (106) passes through the outer wall of the rotating plate (104) and the telescopic rod (500) and is connected to an extension pipe (801). An internally pushed multi-stage moving bladder (802) is fixedly connected inside the telescopic rod (500). The air outlet of the extension pipe (801) is connected to the internally pushed multi-stage moving bladder (802).

8. The facial motion capture method based on deep learning according to claim 7, characterized in that: The base (501) has a sliding groove inside, and a T-shaped slider (600) is slidably connected inside the sliding groove. A fastening band (601) is fixedly connected to the bottom of the T-shaped slider (600).

9. The facial motion capture method based on deep learning according to claim 8, characterized in that: A suction cup (700) is fixedly connected to the bottom of the base (501).

10. A facial motion capture method based on deep learning according to claim 9, characterized in that: Multiple second strain gauges (205) are fixedly connected to the outside of the central support bladder (101), and multiple first strain gauges (109) are fixedly connected to the outside of both the inner pushing airbag (200) and the outer peripheral support bladder (102). The first strain gauges (109) and the second strain gauges (205) are used to generate resistance changes when the central support bladder (101) and the outer peripheral support bladder (102) expand and extend. The angular position of the camera (103) is calculated by the resistance change and the expansion amount. The calculation formula is as follows: ; in, The tilt angle of the outer supporting bladder (102); The original length of the peripheral support capsule (102); The initial resistance of the first strain gauge (109); The airbag deformation coefficient; The volume of gas entering the internal thrust airbag (200); The change in resistance of the first strain gauge (109); The formula relating gas volume to capsule extension is as follows: ,in, For deformable variables, This is the proportionality coefficient. The driving quantity is the input variable; After obtaining the above relational variables, substitute them into the geometric tilt model. The specific formula of the geometric tilt model is: ; in, The tilt angle of the peripheral bearing bladder (102) The cosine value; This refers to the extension of the anterior end of the cyst.

Images