Fully-formed cap construction method based on BiSeNet face segmentation

Abstract

The invention relates to the technical field of formed caps, in particular to a fully-formed cap construction method based on BiSeNet face segmentation, and the method comprises the steps: constructing a face segmentation data set; segmenting the contours of the face, the head and the neck by using the face segmentation model to obtain contour pixel coordinates; calculating the distance between the feature points to obtain hat making parameters; inputting the feature points and the distances between the feature points into model making software to obtain a hat platemaking; according to the invention, the BiSeNet is used for extracting the coordinates of each point on the face contour, and the Euclidean distance, the Harmann distance and the arc length algorithm are respectively used for calculating the length between the feature points, so that a complete and accurate hat model is constructed.

Description

Technical Field

[0001] The present invention relates to the technical field of molded caps, and in particular to a method for constructing a fully molded cap based on BiSeNet face segmentation. Background Art

[0002] The creation of a fully fashioned hat model involves detailed modeling and realistic shape reconstruction of each hat part; traditional methods usually require complex manual adjustments and high-precision scanning equipment, which are not only costly but also inflexible in practical applications.

[0003] Jungil Jun et al. personalized the design baseline by modeling the 3D head scans of people, and developed prototype hat patterns suitable for people according to four hat types. The 3D head scanning equipment and related modeling software under this method are often expensive. In addition, the personalized design and production process may also increase manufacturing costs, which may not be economical for some small businesses or projects with limited budgets.

[0004] Fang Shengyi and others used image processing technology to automatically extract the head and recognize facial features based on three-dimensional geometric relationships, converting time-consuming and complex three-dimensional geometric problems into simple two-dimensional operations; this method achieves three-dimensional to two-dimensional mapping through three-dimensional geometric relationships, which often loses some feature information and reduces generalization ability.

[0005] The publication number is CN 118441404A. The brim size is calculated. The opening structure infers the weaving size according to the arc length formula. The opening brim is divided into three sections according to the characteristic size of the face. However, this method has the following shortcomings:

[0006] 1. The differences in facial features between different individuals are increasing, and relying solely on specific calculation methods cannot meet the needs of all users;

[0007] 2. The braiding size inferred only from the arc length formula fails to fully consider the elasticity and deformation characteristics of the material in actual wearing, resulting in insufficient comfort of the open structure;

[0008] 3. Although the design of dividing the open hat brim into three sections can adapt to a certain range of facial features, it cannot meet the needs of faces of extreme sizes or shapes, resulting in insufficient envelopment and fixation of the hat. Summary of the invention

[0009] In view of the shortcomings of the existing methods, the present invention uses BiSeNet to extract the coordinates of each point on the face contour, and uses Euclidean distance, Harmanton distance, and arc length algorithm to calculate the length between each feature point, so as to construct a complete and accurate hat model.

[0010] The technical solution adopted by the present invention is: a method for constructing a fully fashioned cap based on BiSeNet face segmentation comprises the following steps:

[0011] Step 1: Build a face segmentation dataset;

[0012] Step 2: Use the face segmentation model to segment the contours of the face, head, and neck to obtain the contour pixel coordinates;

[0013] As a preferred implementation of the present invention, the face segmentation model is BiSeNet.

[0014] Step 3: Calculate the distance between each feature point to obtain the hat making parameters;

[0015] As a preferred embodiment of the present invention, the distances between the feature points include: head circumference, neck circumference, shoulder and neck circumference, head length circumference, sagittal arc, face height circumference, chin-nose arc, head and neck length, vertical length of the face, upper face height, nose-frontal length, zygomatic-temporal length, distance between the outer canthi of both eyes, zygomatic arch width, fronto-temporal length, chin-zygomatic length, vertical length from the temporal bone to the middle face, vertical length from the temporal bone to the hairline arc, and vertical length from the zygomatic bone to the submental arc.

[0016] As a preferred embodiment of the present invention, the zygomaticotemporal length, the diameter of the head circumference, the radius of the shoulder and neck circumference, and the vertical length from the temporal bone to the midface are calculated by Euclidean distance.

[0017] As a preferred embodiment of the present invention, the head and neck length, nose-forehead length, distance between the outer canthi of both eyes, zygomatic arch width, chin-nasal arc, upper face height, facial vertical length, fronto-temporal length, and chin-zygomatic length are calculated using the Harmanton distance.

[0018] As a preferred embodiment of the present invention, the calculation of the face height circumference, sagittal arc and head length circumference is to input the starting point, middle point and end point on the face height circumference, sagittal arc and head length circumference into a quadratic equation, obtain the coefficients of the quadratic equation, and use the quadratic equation coefficients to calculate the arc length.

[0019] As a preferred embodiment of the present invention, the formula for calculating the arc length is:

[0020]

[0021] Among them, a and b are the coefficients of the quadratic equation, and x1, x2, and x3 are the x-axis coordinates of the starting point, the middle point, and the end point respectively.

[0022] As a preferred embodiment of the present invention, it includes: a memory for storing instructions executable by a processor; a processor for executing the instructions to implement a method for constructing a fully molded cap based on BiSeNet face segmentation.

[0023] As a preferred embodiment of the present invention, a computer-readable medium storing computer program code is characterized in that the computer program code, when executed by a processor, implements a method for constructing a fully molded cap based on BiSeNet face segmentation.

[0024] Beneficial effects of the present invention:

[0025] 1. Improve segmentation accuracy: The present invention uses deep learning technology to improve the accuracy of facial features segmentation. Compared with traditional image processing methods, it can extract facial features more accurately and ensure that the design of the hat better meets the actual needs of users;

[0026] 2. Simplified calculation process: Through simplified numerical calculation, the present invention can efficiently obtain the parameters required for hat design, reduce complex geometric calculations, and improve the efficiency of design and manufacturing;

[0027] 3. Enhanced applicability: The method of the present invention can process head data of different individuals, has strong versatility and adaptability, and meets the personalized needs of more users;

[0028] 4. Reduce costs: Compared with existing 3D scanning and modeling technologies, the method of the present invention reduces the reliance on expensive equipment and complex software, thereby reducing overall costs and improving economic efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the method for constructing a fully fashioned cap based on BiSeNet face segmentation of the present invention;

[0030] Figure 2 It is the BiSeNet network structure diagram of the present invention;

[0031] Figure 3 It is a schematic diagram of the feature points of the face model;

[0032] Figure 4 is a loss function graph of the present invention;

[0033] Figure 5 This is a comparison chart of the loss functions of existing models;

[0034] Figure 6 It is a comparison diagram of the segmentation results of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0036] like Figure 1As shown, the method for constructing a fully molded cap based on BiSeNet face segmentation includes the following steps:

[0037] Step 1: Build a face segmentation dataset;

[0038] The dataset in this embodiment is CelebAMask-HQ, which contains 30,000 high-resolution celebrity facial images and also provides face segmentation labels; including 19 facial regions, such as skin, eyes, lips, hair, etc.

[0039] Since the dataset does not include facial images from Asia, the imbalance in racial proportions may cause the model to perform poorly when processing Chinese faces, resulting in a decrease in recognition accuracy. The present invention also adopts a multi-channel data acquisition strategy, including collecting Chinese face datasets from the CASIA-WebFace public dataset, the Megvii (Megvii Technology) commercial dataset, and self-collecting Chinese face datasets, using a camera or mobile phone to take 45-degree front and side images of the face, and using software to annotate the dataset to obtain manually segmented masks as model input. Through these approaches, a total of 1,086 Chinese face images were obtained. After eliminating some images that did not conform to reality, 472 images to be segmented were obtained, including 392 frontal photos and 80 side photos.

[0040] Step 2: Build a BiSeNet face segmentation model, use BiSeNet to segment the contours of the face, head, and neck, and obtain the corresponding contour pixel coordinate values;

[0041] The contour coordinate values ​​correspond to the coordinates of the feature points of the face, head, and neck; in this implementation, the coordinates are all pixel coordinates.

[0042] like Figure 2 As shown, BiSeNet consists of two main branches: the spatial path and the context path, which are used to extract local details and global context information of the image respectively;

[0043] The BiSeNet network mainly consists of two main parts: the spatial path and the context path, as well as the fusion of the two paths; BiSeNet uses these two paths to balance the acquisition of detail information and global context information, thereby improving the accuracy of the segmentation task;

[0044] 1. Space path:

[0045] This path is mainly used to preserve the high-resolution details of the image; Figure 2The left part shows how spatial features are gradually extracted through three convolutional layers (Conv+BN+ReLU). The convolution kernel of each layer focuses on local features, especially in segmentation tasks, accurately extracting information of key parts such as facial features. It is combined with the features of the context path through the feature fusion module (FFM) to form a more complete segmentation result.

[0046] 2. Context path:

[0047] The path is located at Figure 2 , which is used to extract the global context information of the image; global features are gradually obtained by downsampling the image layer by layer (4x down, 8x down, 16x down, 32x down); after each downsampling, these global features are further optimized through the ARM (Attention Refinement Module). In ARM, the global context information of the entire feature map is first obtained through the global pooling layer, and then a weight matrix is ​​generated through a 1x1 convolution layer, batch normalization and sigmoid activation function to enhance meaningful features.

[0048] 3. Feature Fusion:

[0049] The features of the context path and the spatial path are combined in the feature fusion module. Specifically, the high-resolution features of the spatial path are combined with the global features of the context path through ARM processing, which adds weights to the features of both parts and restores them to the original resolution through upsampling (8x upsample). In the fusion stage, the features of the two paths are combined through a concatenation operation, and then the fused features are further processed through a convolution operation to ensure a balance between details and global information.

[0050] 4. Global pooling and final output:

[0051] Figure 2 The right side shows the global pooling operation in the context path, which obtains the global information of the image through a global pooling layer, and then adjusts the weight of the feature through a 1x1 convolution layer and a sigmoid function. This step ensures that the model can effectively extract the global semantic information of the image.

[0052] Step 3: Calculate the distance between each feature point, that is, obtain the hat making parameters by calculating the distance between the feature points of the facial features;

[0053] like Figure 3 As shown, in order to obtain the parameters for generating the hat model,

[0054] Step 31, first calculate the dimensional parameters of the head, including: head circumference d1, neck circumference d2, shoulder and neck circumference d3, head length circumference d4, sagittal arc d5, face height circumference d6 and chin-nose arc d7;

[0055] The head circumference d1 is the circumference from point G to point J, wherein point G is the center of the eyebrows, and point J is the back of the head point.

[0056] The neck circumference d2 is the horizontal circumference of the middle part of the neck, that is, the circumference from point O to point O', where point O is the Adam's node point and point O' is the fourth cervical vertebra point.

[0057] The shoulder and neck circumference d3 is the horizontal circumference of the neck and shoulder side points, that is, the circumference from point C to point B. Point C is the lowest end of the front neck, and point B is the intersection of the neck and shoulders.

[0058] The head length circumference d4 is the circumference from point G, point H, point I, point J to point K; point H is the hairline point; point I is the top of the head; point K is the nape of the neck.

[0059] The sagittal arc d5 is the circumference from point H, point I, point J to point K.

[0060] The face height circumference d6 is the circumference from point D, point P to point F; point D is the submental point, point F is the mid-point of the nose, and point P is the tip of the nose.

[0061] The chin-nose arc d7 is the circumference from point D to point E; point E is the subnasal point.

[0062] Step 32, calculating the height parameters of the head, including: head and neck length h1, facial vertical length h2, upper facial height h3, nose-frontal length h4 and zygomatic-temporal length h5;

[0063] The head and neck length h1 is the length from point I to point C.

[0064] The vertical length of the face h2 is the length from point H to point D.

[0065] The height h3 of the upper surface is the length from point H to point E.

[0066] The nose-forehead length h4 is the length from point G to point F.

[0067] The zygomaticotemporal length h5 is the length from point L to point M; point L is the lateral temporal bone point; point M is the right zygomatic bone point.

[0068] Step 33, calculating the lateral parameters of the head, including: the distance between the outer canthi of both eyes w1, the width of the zygomatic arch w2;

[0069] The distance w1 between the outer canthi of both eyes is the horizontal distance between point N and point N'; point N is the right outer point of the eye corner, and point N' is the left outer point of the eye corner.

[0070] The zygomatic arch width w2 is the horizontal distance between point M and point M'; point M' is the left point of the zygomatic bone.

[0071] Step 34, calculating the oblique parameters of the head, including: the fronto-temporal length b1 and the chin-zygomatic length b2;

[0072] The frontotemporal length b1 is the oblique length from point H to point L.

[0073] The chin-zygomatic length b2 is the oblique length from point D to point M.

[0074] Step 35, calculating the inclination parameters of the head, including: the vertical length a1 from the temporal bone to the midface, the vertical length a2 from the temporal bone to the hairline arc, and the vertical length a3 from the zygomatic bone to the submental arc;

[0075] The vertical length a1 from the temporal bone to the midface is the length from point L to the midpoint of line segment EH.

[0076] The vertical length a2 from the temporal bone to the hairline is the maximum vertical length from the midpoint of the line segment LH to the arc LH, which is fixed at 2 cm.

[0077] The vertical length a3 from the zygomatic bone to the submental arc is the maximum vertical length from the midpoint of the line segment MD to the arc line MD, which is fixed at 2 cm.

[0078] Among them, the zygomaticotemporal length h5, the diameter of the head circumference d1, the radius of the shoulder and neck circumference d3, and the vertical length from the temporal bone to the midface a1 are calculated using the Euclidean distance, and the formula is:

[0079]

[0080] Among them, ρ is the Euclidean distance between the pixel values ​​of h5, d1, d3, and a1.

[0081] When calculating h5, (x1, y1) (x2, y2) are the coordinates of point L and point M respectively; when calculating d1, (x1, y1) (x2, y2) are the coordinates of point G and point J respectively. Let the diameter of d1 be ρ d1 , through the circle formula πρ d1 Calculate the circumference of d1; when calculating d3, (x1, y1)(x2, y2) are the coordinates of points C and B respectively, let the diameter of d3 ρd3, through the circumference formula 2πρ d3 Obtain the circumference of d3; when calculating a1, (x1, y1)(x2, y2) are the coordinate values ​​of the midpoints of points L and EH respectively.

[0082] When calculating the circumference of the neck circumference d2, since the coordinates of point O and O' cannot be accurately obtained, first use point C and point D to calculate the ordinate y1 of point O. Among them, y C ,y D is the ordinate of point C and D; secondly, the horizontal coordinate x1 of point O is obtained according to y1 on the coordinate sequence of the front neck segmentation curve; thirdly, the horizontal coordinate x of point B is obtained according tob Approximately the midpoint coordinate of OO', |x1-x b | is the radius of OO'; finally, using the circle formula 2π|x1-x b |Calculate d2.

[0083] The lengths of h1, h4, w1, w2, d7, h3, h2, b1, and b2 are calculated using the Harmanton distance, using the following formula:

[0084] ρ=|x'1-x'2|+|y'1-y'2| (3)

[0085] Where ρ is the Harmanton distance of the pixel values ​​of h1, h4, w1, w2, d7, h3, h2, b1, and b2.

[0086] When calculating h1, (x'1, y'1) (x'2, y'2) are the coordinates of point I and point C respectively; when calculating h4, (x'1, y'1) (x'2, y'2) are the coordinates of point G and point F respectively; when calculating w1, (x'1, y'1) (x'2, y'2) are the coordinates of point N and N' respectively; when calculating w2, (x'1, y'1) (x'2, y'2) are the coordinates of point M and M' respectively; when calculating d7 ... F respectively. '2,y'2) are the coordinate values ​​of point D and point E respectively; when calculating h3, (x'1,y'1)(x'2,y'2) are the coordinate values ​​of point H and point E respectively; when calculating h2, (x'1,y'1)(x'2,y'2) are the coordinate values ​​of point H and point D respectively; when calculating b1, (x'1,y'1)(x'2,y'2) are the coordinate values ​​of point H and point L respectively; when calculating b2, (x'1,y'1)(x'2,y'2) are the coordinate values ​​of point D and point M respectively.

[0087] d6, d4, and d5 are calculated by arc length. When calculating d6, select the end point D of the side point, the middle point F, and the end point P. Set the coordinates of the three points to (x1, y1)(x2, y2)(x3, y3);

[0088] When calculating d4, select the starting point G and the end point K among points G, H, I, J, and K, and any point among the middle points H, I, and J with coordinates (x1, y1)(x2, y2)(x3, y3);

[0089] When calculating d4, select the coordinates of any point among the side points H, I, J, and K, the starting point H and the end point K, and the middle points I and J as (x1, y1)(x2, y2)(x3, y3);

[0090] Taking d6 as an example, substitute the coordinates of the three points into the function y=ax 2 +bx+c, find the coefficients a, b, c;

[0091]

[0092] Calculate the arc length L of d6 using the formula:

[0093]

[0094] but

[0095]

[0096] Among them, a, b, c are the coefficients of the quadratic equation.

[0097] a2 and a3 are custom parameters, which are set to 2cm in this example.

[0098] Step 4: Input each feature point and the distance between each feature point into the model making software to obtain the hat pattern;

[0099] By calculating the key distance between the head and the facial features, the parameters such as head circumference, neck mid-circumference, upper circumference and lower circumference required for the fully molded hat can be accurately obtained; the calculated feature points and the corresponding distances are transmitted to the model making software for the production of supporting hats, such as CAD design software, which will have corresponding plate making operations. The plate making of the hat is realized by combining the data with the CAD software, and the hat is completed through the mechanical equipment of the computer flat knitting machine, which provides a scientific basis for the design of the final hat model and ensures its accuracy and comfort in size and fit.

[0100] Experimental process:

[0101] The bidirectional neural network BiSeNet model is used to further improve the speed by processing two branches simultaneously to achieve face semantic segmentation. The experimental environment is: Windows 10 operating system, 16GB memory, RTX3060 graphics card, 12GB video memory, IDE is PyCharm; the input image is 512*512, and a total of 200 rounds of training are conducted. The loss function of the validation set is as follows Figure 4 As shown in the figure, in these applications, processing speed is a key factor; although the traditional FCN and DeepLab models have good performance in accuracy, they are relatively slow in inference speed, especially in the processing of high-resolution images; the fully molded hat model constructed for the human head model of the present invention often pays more attention to the processing speed of the model. Due to the characteristics of knitted fabrics, the accuracy requirements are relatively high, so the BiSeNet network is selected for implementation; the model losses of the three models are not much different under the condition of two hundred rounds of training. The loss comparison of the FCN and DeepLab models is as follows Figure 5 As shown; Comparison of segmentation of three models Figure 4 ,5 : BiSeNet, Deeplab, Fcn, respectively. Figure 6 It can be verified that BiSeNet can better segment the global semantics. Although the boundary information is not as clear as the other two networks, the segmentation of global semantics improves the accuracy of hat parameter calculation.

[0102] The captured images are manually measured to calculate the actual values ​​of the parameters required for the distance between the above facial feature points. Since the distance calculated by the model is calculated based on the distance of the pixel values ​​on the screen, there is a fixed ratio with the actual distance when the camera parameters and shooting distance are fixed. The actual value can be obtained by finding the ratio through experiments and inserting it into the formula. The comparison results are shown in Table 1 below:

[0103] Table 1 Calculation of facial features parameter model values ​​based on different student fixed ratio parameter values

[0104]

[0105] It can be seen from Table 1 that under the shooting conditions, including shooting distance, lighting, and fixed equipment, pixel value distance = measured value * 0.6, and the difference between the automatic value and the pixel value distance is the error.

[0106] Table 2 verifies and compares the ratio 0.6 obtained in Table 1

[0107]

[0108]

[0109] The values ​​automatically output by the model are calculated according to the formula pixel value distance = true distance * 0.6 to obtain the converted data. The error value of each item is obtained by subtracting the converted data from the measured value. From Table 2, it can be seen that the maximum model error is 2.883 cm, which meets the hat generation standard and does not affect the comfort.

[0110] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for constructing a fully fashioned cap based on BiSeNet face segmentation, characterized in that: Step 1: Build a face segmentation dataset; Step 2: Use the face segmentation model to segment the contours of the face, head, and neck to obtain the contour pixel coordinates; Step 3: Calculate the distance between each feature point to obtain the hat making parameters; Step 4: Input each feature point and the distance between each feature point into the model making software to obtain the hat pattern.

2. The method for constructing a fully fashioned cap based on BiSeNet face segmentation according to claim 1, characterized in that: The distances between the feature points include: head circumference, neck circumference, shoulder and neck circumference, head length circumference, sagittal arc, face height circumference, chin-nose arc, head and neck length, vertical length of the face, upper face height, nose-frontal length, zygomatic-temporal length, distance between the outer canthi of both eyes, zygomatic arch width, fronto-temporal length, chin-zygomatic length, vertical length from the temporal bone to the midface, vertical length from the temporal bone to the hairline arc, and vertical length from the zygomatic bone to the submental arc.

3. The method for constructing a fully fashioned cap based on BiSeNet face segmentation according to claim 2, characterized in that: The zygomaticotemporal length, the diameter of the head circumference, the radius of the shoulder and neck circumference, and the vertical length from the temporal bone to the midface were calculated using the Euclidean distance.

4. The method for constructing a fully fashioned cap based on BiSeNet face segmentation according to claim 2, characterized in that: The head and neck length, nose-forehead length, distance between the outer canthi of both eyes, zygomatic arch width, chin-nasal arc, upper face height, vertical face length, fronto-temporal length, and chin-zygomatic length were calculated using the Harmanton distance.

5. The method for constructing a fully fashioned cap based on BiSeNet face segmentation according to claim 2, characterized in that: The calculation of face height circumference, sagittal arc and head length circumference is to input the starting point, middle point and end point on the face height circumference, sagittal arc and head length circumference into a quadratic equation, obtain the coefficients of the quadratic equation, and use the coefficients of the quadratic equation to calculate the arc length.

6. The method for constructing a fully fashioned cap based on BiSeNet face segmentation according to claim 5, characterized in that: The formula for calculating arc length is: Among them, a and b are the coefficients of the quadratic equation, and x1, x2, and x3 are the x-axis coordinates of the starting point, the middle point, and the end point respectively.

7. The method for constructing a fully fashioned cap based on BiSeNet face segmentation according to claim 1, characterized in that: The face segmentation model is BiSeNet.

8. A fully formed cap construction system based on BiSeNet face segmentation, characterized in that: include: a memory for storing instructions executable by a processor; A processor, used for executing instructions to implement the method for constructing a fully molded cap based on BiSeNet face segmentation as described in any one of claims 1 to 7.

9. A computer readable medium storing a computer program code, characterized in that: The computer program code, when executed by a processor, implements the method for constructing a fully molded cap based on BiSeNet face segmentation as described in any one of claims 1 to 7.

Images