A method for facial reconstruction of curling players based on broadcast video

Abstract

This paper proposes a method for reconstructing the faces of curling athletes based on broadcast videos. This method optimizes the facial features of curling athletes in broadcast videos and utilizes deep learning techniques combined with a mask network strategy to overcome the limitations of traditional methods in processing reflective and weakly textured areas. This method provides more realistic and vivid digital human images for applications such as the metaverse, digital humans, and sports broadcasts.

Description

Technical Field

[0001] The present invention relates to the fields of metaverse, digital humans, and artificial intelligence technology, and in particular to a method for reconstructing the face of a curling player based on broadcast video. Background Art

[0002] In curling match broadcasts, athletes' facial expressions and interactions are crucial for creating an immersive and engaging experience for viewers. However, due to the effects of venue lighting and ice reflections during curling matches, the texture information on athletes' faces is particularly complex and difficult to capture.

[0003] Traditional 3D modeling methods, such as manual software modeling, high-precision instrument scanning, and multi-view image fusion, can achieve good results in specific scenarios, but they often suffer from high equipment costs, complex operational procedures, and strict dependence on environmental conditions. Multi-view image fusion modeling, in particular, while offering excellent reconstruction accuracy and edge processing, often suffers from model incompleteness and texture loss when processing the highly reflective and weakly textured facial expressions of curling athletes in broadcast video. This severely impacts the fidelity and practicality of the curling athlete digital images. Summary of the Invention

[0004] This invention aims to address the challenges of the existing technology by proposing a method for reconstructing the faces of curling athletes based on broadcast videos. This method considers that weak texture regions in the curling athletes' facial images in broadcast videos primarily occur on the cheeks, forehead, and hair. By extracting the correct weak texture regions from the curling athletes' facial images, and combining traditional multi-view 3D reconstruction with deep learning multi-view 3D reconstruction methods, an evaluation-based reconstruction method is proposed, improving the integrity and usability of multi-view facial 3D reconstruction.

[0005] The present invention is implemented by the following technical solution. The present invention proposes a method for reconstructing the face of a curling player based on a broadcast video. The method comprises the following steps:

[0006] Step 1: Collect multi-view images of the curler during the throwing, sliding, rubbing, and decision-making phases of the broadcast video. Calculate the internal and external parameters of the broadcast camera at different stages. Extract key points on the curler's face and identify the facial features. Use a weak texture detection algorithm to extract weak texture areas from the rest of the face and construct a facial depth map merge mask.

[0007] Step 2: Use a depth estimation method to obtain a first depth map from multiple perspectives, use a deep learning depth estimation method to obtain a second depth map from multiple perspectives, align and normalize the first and second depth maps, and merge them according to the mask to obtain a third depth map;

[0008] Step 3: Fuse the third depth maps from multiple perspectives and use a multi-perspective reconstruction method to obtain a facial 3D model with texture mapping. Quantitatively evaluate the reconstruction effect of the facial 3D model based on the model smoothness evaluation method and the model integrity evaluation method. Update the depth map merging mask based on the evaluation results until a complete and correct curling athlete's facial model is reconstructed.

[0009] Furthermore, in step 1, the key points of the curling player's face are extracted, and the facial features are determined as follows: multi-perspective images of the curling player in the throwing stage, sliding stage, ice-wiping stage and decision-making stage are collected from the broadcast video, and the non-facial parts are replaced with a gray background; by matching the image feature points, the matches that meet the geometric constraints are retained, and the internal and external parameters of the broadcast camera are restored using the SFM method; the key points of the facial features, including eyebrows, eyes, nose and mouth, are extracted, the convex hull is drawn and feathered to obtain the facial features. When the facial features are removed, the weak texture areas of the cheeks, forehead and hair are obtained.

[0010] Furthermore, in step 1, the weak texture area is extracted based on the grayscale gradient threshold method as follows: the horizontal and vertical Prewitt operator convolution is used to obtain the gradient g in the x and y directions of the pixel point x (x,y),g y (x,y), then the gradient value of this pixel is Where (x, y) is the pixel position; if this pixel belongs to a weak texture area, then it satisfies: g(x, y) < g t , where g t is the gradient threshold, and the obtained pixel position is the segmented weak texture area.

[0011] Furthermore, in step 1, the mask for depth map merging is obtained as follows: after detecting the weak texture area in the facial image, the mask value of the weak texture area in the image is set to 1, and the mask value of the rest of the image is set to 0, that is:

[0012]

[0013] Among them, face_mask(x,y) is the depth merge mask.

[0014] Furthermore, the step 2 is specifically as follows:

[0015] Step 2.1: Use the PatchMatch method to perform depth estimation. Taking the broadcast camera pose and the curling athlete's facial image as input, the first depth map, depthmap1, is obtained through the matching cost construction, accumulation, estimation, and optimization process. This effectively distinguishes the background from the facial area. The deep learning network PVA-MVSNet is used to aggregate multi-scale pyramid image information based on multi-metrics. This multi-scale information is used to estimate the depth value of the weak texture area, resulting in the second depth map, depthmap2.

[0016] Step 2.2: Align and normalize the first depth map and the second depth map; normalize the first depth map to the depth range of the second depth map, and design the normalization coefficient as:

[0017]

[0018] Among them, Max{} refers to the maximum depth value in the depth map, and Min{} refers to the minimum depth value in the depth map;

[0019] The normalized first depth map Depthmap1 is:

[0020] Depthmap1=Min{depthmap2}+k*(depthmap1-Min{depthmap1});

[0021] Step 2.3: Merge the first depth map and the second depth map according to the mask to obtain a third depth map depthmap3, and calculate the depth difference at the edge. If it exceeds the threshold, perform weighted filtering to smooth the depth map; the merging method is:

[0022] depthmap3=face_mask*depthmap2+(1-face_mask)*Depthmap1

[0023] The method for judging the depth difference at the edge is:

[0024] ∫|Depthmap1(edge)-depthmap2(edge)| <th

[0025] Among them, edge is the edge area of ​​the depth merging mask, th is the depth difference threshold; the weighted filtering method selects the convolution sliding average filtering algorithm with a window size of 5*5.

[0026] Furthermore, in step 3, the multi-perspective third depth maps of the curling player in the throwing phase, gliding phase, ice-wiping phase and decision-making phase in the broadcast video are integrated, and a dense point cloud set of the curling player's facial model in the three-dimensional space is calculated. The point cloud set is tetrahedral meshing and surface extraction is performed to obtain a facial mesh model, and a mapping-based color map mesh is used to generate a texture-mapped three-dimensional facial model of the curling player.

[0027] Furthermore, in step 3, the model smoothness evaluation is specifically as follows: the Gaussian curvature of the vertices of the facial 3D model excluding the facial features is calculated:

[0028]

[0029] Among them, k G is the Gaussian curvature, v i is the i-th vertex, N(i) is the number of all vertex v i The number of triangles, θ ij is the angle difference between the jth triangle patch and the vertex normal vector;

[0030] If the number of points whose Gaussian curvature is greater than the curvature threshold is greater than the number threshold, the depth value of the edge area changes greatly when the depth map is merged. In this case, the window size of the weighted filter is increased.

[0031] Furthermore, in step 3, the model integrity evaluation is specifically as follows: a virtual camera is constructed according to the internal and external parameters of the broadcast camera calculated in step 1, images of the curling athlete's facial texture model from different perspectives are rendered, and the structural similarity (SSIM) index and peak signal-to-noise ratio (PSNR) index of each perspective image and the image captured by the broadcast camera are calculated;

[0032] Specifically, the structural similarity SSIM index is

[0033]

[0034] Among them, μ x ,μ y are the average values ​​of x and y respectively, are the variances of x and y, σ xy is the covariance of x and y; c1 and c2 are constants used to maintain stability;

[0035] For a given original object of size m*n Figure I And the noise map K, the mean square error is expressed as follows:

[0036]

[0037] The peak signal-to-noise ratio PSNR indicator is: Among them MAX Iis the maximum pixel value of the image;

[0038] When the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) indicators are lower than the threshold, the evaluation model is incomplete. In this case, the non-zero part in the depth map merge mask is expanded or contracted to update the depth merge mask. Dilation of the non-zero part optimizes the incomplete model caused by weak texture. Contraction of the non-zero part optimizes the incomplete model caused by edge performance and accuracy issues.

[0039] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for reconstructing the face of a curling player based on a broadcast video are implemented.

[0040] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the method for reconstructing the face of a curling player based on broadcast video.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This paper proposes a method for reconstructing the faces of curling athletes based on broadcast videos. This method effectively addresses the issues of incomplete models and missing textures encountered by traditional 3D modeling methods when dealing with highly reflective and weakly textured faces. By combining deep learning techniques with mask network strategies, this method optimizes the reconstruction of curling athletes' facial features in broadcast videos, significantly improving the realism and practicality of digital human images. This advancement brings more realistic and vivid visual effects to fields such as the metaverse, digital humans, and sports broadcasting, while reducing reliance on high-cost equipment and complex operational processes and improving modeling efficiency and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 This is a flow chart of a curling player's facial reconstruction method based on broadcast video described in the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] Combine Figure 1 The present invention proposes a method for reconstructing the face of a curling player based on a broadcast video, the method comprising the following steps:

[0047] Step 1: Collect multi-view images of the curler during the throwing, sliding, rubbing, and decision-making phases of the broadcast video. Calculate the internal and external parameters of the broadcast camera at different stages. Extract key points on the curler's face and identify the facial features. Use a weak texture detection algorithm to extract weak texture areas from the rest of the face and construct a facial depth map merge mask.

[0048] In step 1, the key points of the curling players' faces are extracted and the facial features are determined. Specifically, multi-perspective images of the curling players in the throwing, sliding, rubbing and decision-making stages are collected from the broadcast video, and the non-facial parts are replaced with a gray background. By matching the image feature points and retaining the matches that meet the geometric constraints, the internal and external parameters of the broadcast camera are restored using the SFM method. The key points of the facial features, including eyebrows, eyes, nose and mouth, are extracted, and the convex hull is drawn and feathered to obtain the facial features. When the facial features are removed, weak texture areas such as cheeks, forehead and hair can be obtained.

[0049] In step 1, the weak texture area is extracted based on the grayscale gradient threshold method: the horizontal and vertical Prewitt operator convolution is used to obtain the gradient g in the x and y directions of the pixel point. x (x,y),g y (x,y), then the gradient value of this pixel is Where (x, y) is the pixel position; if this pixel belongs to a weak texture area, then it satisfies: g(x, y) < g t , where g t is the gradient threshold, and the obtained pixel position is the segmented weak texture area.

[0050] In step 1, the mask for depth map merging is obtained as follows: after detecting the weak texture area in the facial image, the mask value of the weak texture area in the image is set to 1, and the mask value of the rest of the image is set to 0, that is:

[0051]

[0052] Among them, face_mask(x,y) is the depth merge mask.

[0053] Step 2: Use the traditional depth estimation method to obtain the first depth map of multiple views, use the deep learning depth estimation method to obtain the second depth map of multiple views, align and normalize the first depth map and the second depth map, and merge them according to the mask to obtain the third depth map;

[0054] The step 2 is specifically as follows:

[0055] Step 2.1: Use the PatchMatch method to perform depth estimation. Taking the broadcast camera pose and the curling athlete's facial image as input, the first depth map, depthmap1, is obtained through the matching cost construction, accumulation, estimation, and optimization process. This effectively distinguishes the background from the facial area. The deep learning network PVA-MVSNet is used to aggregate multi-scale pyramid image information based on multi-metrics. This multi-scale information is used to estimate the depth value of the weak texture area, resulting in the second depth map, depthmap2.

[0056] Step 2.2: Align and normalize the first depth map and the second depth map; normalize the first depth map to the depth range of the second depth map, and design the normalization coefficient as:

[0057]

[0058] Among them, Max{} refers to the maximum depth value in the depth map, and Min{} refers to the minimum depth value in the depth map;

[0059] The normalized first depth map Depthmap1 is:

[0060] Depthmap1=Min{depthmap2}+k*(depthmap1-Min{depthmap1});

[0061] Step 2.3: Merge the first depth map and the second depth map according to the mask to obtain a third depth map depthmap3, and calculate the depth difference at the edge. If it exceeds the threshold, perform weighted filtering to smooth the depth map; the merging method is:

[0062] depthmap3=face_mask*depthmap2+(1-face_mask)*Depthmap1

[0063] The method for judging the depth difference at the edge is:

[0064] ∫|Depthmap1(edge)-depthmap2(edge)| <th

[0065] Among them, edge is the edge area of ​​the depth merging mask, th is the depth difference threshold; the weighted filtering method selects the convolution sliding average filtering algorithm with a window size of 5*5.

[0066] Step 3: Fuse the third depth maps from multiple perspectives and use a multi-perspective reconstruction method to obtain a facial 3D model with texture mapping. Quantitatively evaluate the reconstruction effect of the facial 3D model based on the model smoothness evaluation method and the model integrity evaluation method. Update the depth map merging mask based on the evaluation results until a complete and correct curling athlete's facial model is reconstructed.

[0067] In step 3, the multi-perspective third depth maps of the curling player in the throwing phase, sliding phase, ice-wiping phase and decision-making phase in the broadcast video are integrated to calculate the dense second point cloud set of the curling player's facial model in the three-dimensional space. The second point cloud set is tetrahedral meshing and surface extraction is performed to obtain the facial mesh model, and the mapping-based color map mesh is used to generate the three-dimensional facial model of the curling player after texture mapping.

[0068] In step 3, the model smoothness evaluation is specifically as follows: the Gaussian curvature of the vertices of the facial 3D model excluding the facial features is calculated:

[0069]

[0070] Among them, k G is the Gaussian curvature, v i is the i-th vertex, N(i) is the number of all vertex v i The number of triangles, θ ij is the angle difference between the jth triangle patch and the vertex normal vector;

[0071] If the number of points whose Gaussian curvature is greater than the curvature threshold is greater than the number threshold, the depth value of the edge area changes greatly when the depth map is merged. In this case, the window size of the weighted filter is increased.

[0072] In step 3, the model integrity evaluation is specifically as follows: a virtual camera is constructed according to the internal and external parameters of the broadcast camera calculated in step 1, images of the curling athlete's facial texture model from different perspectives are rendered, and the structural similarity (SSIM) index and peak signal-to-noise ratio (PSNR) index of each perspective image and the image taken by the broadcast camera are calculated;

[0073] Specifically, the structural similarity SSIM index is

[0074]

[0075] Among them, μ x ,μ y are the average values ​​of x and y respectively, are the variances of x and y, σxy is the covariance of x and y; c1 and c2 are constants used to maintain stability;

[0076] For a given original object of size m*n Figure I And the noise map K, the mean square error is expressed as follows:

[0077]

[0078] The peak signal-to-noise ratio PSNR indicator is: Among them MAX I is the maximum pixel value of the image;

[0079] When the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) indicators are lower than the threshold, the evaluation model is incomplete. In this case, the non-zero part in the depth map merge mask is expanded or contracted to update the depth merge mask. Dilation of the non-zero part optimizes the incomplete model caused by weak texture. Contraction of the non-zero part optimizes the incomplete model caused by edge performance and accuracy issues.

[0080] This paper proposes a facial reconstruction method for curling athletes based on broadcast videos. The method optimizes the facial features of curling athletes in the broadcast videos, uses deep learning technology, and combines it with a mask network strategy to overcome the limitations of traditional methods in processing reflective and weak texture areas, providing more realistic and vivid digital human images for fields such as the metaverse, digital humans, and sports event broadcasts.

[0081] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for reconstructing the face of a curling player based on a broadcast video are implemented.

[0082] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the method for reconstructing the face of a curling player based on broadcast video.

[0083] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0084] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0085] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0086] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0087] The above is a detailed introduction to the method for reconstructing the face of a curling player based on broadcast video proposed by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for reconstructing curling players' faces based on broadcast video, characterized in that: The method comprises the following steps: Step 1: Collect multi-view images of the curler during the throwing, sliding, rubbing, and decision-making phases of the broadcast video. Calculate the internal and external parameters of the broadcast camera at different stages. Extract key points on the curler's face and identify the facial features. Use a weak texture detection algorithm to extract weak texture areas from the rest of the face and construct a facial depth map merge mask. Step 2: Use a depth estimation method to obtain a first depth map from multiple perspectives, use a deep learning depth estimation method to obtain a second depth map from multiple perspectives, align and normalize the first and second depth maps, and merge them according to the mask to obtain a third depth map; Step 3: The multi-view third depth maps are fused and a multi-view reconstruction method is used to obtain a 3D facial model with texture mapping. The reconstruction effect of the 3D facial model is quantitatively evaluated using the model smoothness evaluation method and the model integrity evaluation method. The depth map merging mask is updated based on the evaluation results until a complete and correct facial model of the curling athlete is reconstructed. The step 2 is specifically as follows: Step 2.1: Use the PatchMatch method to perform depth estimation. Take the broadcast camera pose and the curling athlete's face image as input, and obtain the first depth map through the matching cost construction, accumulation, estimation and optimization process. , effectively distinguishing the background from the facial area; using the deep learning network PVA-MVSNet, based on multi-metric aggregation of multi-scale pyramid image information, using multi-scale information to estimate the depth value of the weak texture area, and obtain the second depth map ; Step 2.2: Align and normalize the first depth map and the second depth map; normalize the first depth map to the depth range of the second depth map, and design the normalization coefficient as: ， in, Max {} refers to the maximum depth value in the depth map, Min {} refers to the minimum depth value in the depth map; Normalized first depth map Depthmap 1 is: Depthmap 1= Min { depthmap 2}+ k *( depthmap 1－ Min { depthmap 1})； Step 2.3: Merge the first depth map and the second depth map according to the mask to obtain a third depth map. depthmap 3. Calculate the depth difference at the edge. If it exceeds the threshold, perform weighted filtering to smooth the depth map. The merging method is: depthmap 3=face_mask* depthmap 2+(1-face_mask)* Depthmap 1 in, is the depth merging mask; The method for judging the depth difference at the edge is: ∫| Depthmap 1(edge)- depthmap 2(edge)|< th Among them, edge is the edge area of ​​the depth merge mask, th is the depth difference threshold; the weighted filtering method selects the convolution sliding average filtering algorithm with a window size of 5*5.

2. The method according to claim 1, characterized in that In step 1, the key points of the curling players' faces are extracted and the facial features are determined. Specifically, multi-perspective images of the curling players in the throwing, sliding, rubbing and decision-making stages are collected from the broadcast video, and the non-facial parts are replaced with a gray background. By matching the image feature points, the matches that meet the geometric constraints are retained, and the internal and external parameters of the broadcast camera are restored using the SFM method. The key points of the facial features, including eyebrows, eyes, nose and mouth, are extracted, and the convex hull is drawn and feathered to obtain the facial features. When the facial features are removed, the weak texture areas of the cheeks, forehead and hair are obtained.

3. The method according to claim 2, characterized in that In step 1, the weak texture area is extracted based on the grayscale gradient threshold method: the horizontal and vertical Prewitt operator convolution is used to obtain the gradient of the pixel point in the x and y directions. , then the gradient value of this pixel is , where (x, y) is the pixel position; if this pixel belongs to a weak texture area, then it satisfies: ,in, is the gradient threshold, and the obtained pixel position is the segmented weak texture area.

4. The method according to claim 3, characterized in that In step 1, the mask for depth map merging is obtained as follows: after detecting the weak texture area in the facial image, the mask value of the weak texture area in the image is set to 1, and the mask value of the rest of the image is set to 0, that is: in, is the depth merging mask.

5. The method according to claim 4, characterized in that In step 3, the multi-perspective third depth maps of the curling player in the throwing phase, sliding phase, ice-wiping phase and decision-making phase in the broadcast video are integrated to calculate the dense point cloud set of the curling player's facial model in the three-dimensional space. The point cloud set is tetrahedral meshing and surface extraction is performed to obtain the facial mesh model. The mapping-based color map mesh is used to generate the three-dimensional facial model of the curling player after texture mapping.

6. The method according to claim 5, characterized in that In step 3, the model smoothness evaluation is specifically as follows: the Gaussian curvature of the vertices of the facial 3D model excluding the facial features is calculated: in, is the Gaussian curvature, It is i vertices, are all the included vertices The number of triangles, is the angle difference between the jth triangle patch and the vertex normal vector; If the number of points whose Gaussian curvature is greater than the curvature threshold is greater than the number threshold, the depth value of the edge area changes greatly when the depth map is merged. In this case, the window size of the weighted filter is increased.

7. The method according to claim 6, characterized in that In step 3, the model integrity evaluation is specifically as follows: a virtual camera is constructed according to the internal and external parameters of the broadcast camera calculated in step 1, images of the curling athlete's facial texture model from different perspectives are rendered, and the structural similarity (SSIM) index and peak signal-to-noise ratio (PSNR) index of each perspective image and the image taken by the broadcast camera are calculated; Specifically, the structural similarity SSIM index is in, They are The average value of They are The variance of yes covariance of It is a constant used to maintain stability; For a given original image of size m*n Ⅰ and noise graph K , the mean square error is expressed as follows: The peak signal-to-noise ratio PSNR indicator is: in is the maximum pixel value of the image; When the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) indicators are lower than the threshold, the evaluation model is incomplete. In this case, the non-zero part in the depth map merge mask is expanded or contracted to update the depth merge mask. Dilation of the non-zero part optimizes the incomplete model caused by weak texture. Contraction of the non-zero part optimizes the incomplete model caused by edge performance and accuracy issues.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images