Head size measurement method and apparatus, head grid generation method and apparatus, medium, and device

Abstract

The present disclosure relates to the technical field of artificial intelligence, and provides a head size measurement method and a 3D head grid generation method. According to the method, a face local area is determined by means of face feature points, a circular reference object is extracted from the face local area, and a head size of a target object is determined on the basis of the size of the circular reference object, thereby generating a 3D head grid. The accuracy is high.

Description

Head size measurement, head grid generation method and device, medium and equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to Chinese patent application number 2023108333083, filed on July 10, 2023, entitled “Head size measurement, head grid generation method and device, medium and equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method and device for measuring head size and generating head grid, a computer storage medium, and an electronic device. Background Art

[0004] Currently, commercial 3D scanners are the most widely used method for obtaining 3D physical head models. However, they are expensive and inconvenient, which limits their popularity and accessibility. Therefore, reconstructing 3D models from 2D images is both cheaper and more convenient, making it more suitable for the public.

[0005] Many established 3D head / face reconstruction methods exist, but they primarily focus on applications where physical dimensional accuracy is less critical. In contrast, dimensionality is crucial in ergonomic design. Current methods are unable to derive 3D head / face models with sufficient dimensional accuracy.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field.

[0007] Summary of the Invention

[0008] The present disclosure aims to provide a head size measurement method, a 3D head mesh generation method and an apparatus thereof, which at least to some extent overcome the problem of inaccurate head size determination in related arts.

[0009] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0010] According to one aspect of the present disclosure, a head size measurement method is provided, comprising: acquiring a facial image of a target subject, the facial image having a circular reference object at the forehead position of the target subject; determining facial feature points of the target subject; determining a local facial region including the circular reference object based on the facial feature points; extracting the circular reference object from the local facial region; and determining the head size of the target subject based on the size of the circular reference object.

[0011] In one embodiment, determining the head size of the target object based on the size of the circular reference object includes: determining a diameter of the circular reference object; and determining the head size of the target object based on the diameter of the circular reference object.

[0012] In one embodiment, the facial feature points include eye feature points, and the facial partial image including the circular reference object is a rectangular area including the circular reference object.

[0013] In one embodiment, extracting the circular reference object from the local facial area includes: segmenting the local facial area using an image segmentation artificial neural network to obtain the circular reference object; detecting the edge of the circular reference object using an edge detector; and performing ellipse measurement on the edge of the circular reference object using an ellipse fitting method, thereby extracting the circular reference object.

[0014] In one embodiment, the image segmentation artificial neural network is a U-Net or a fully convolutional network FCN, and / or the edge detector is a canny edge detector.

[0015] In one embodiment, the method further includes: performing dedistortion processing on the facial image of the target object based on camera parameters of an image acquisition device, wherein the camera parameters include distortion coefficients and an eigenmatrix, and the distortion coefficients include radial distortion coefficients and tangential distortion coefficients.

[0016] In one embodiment, the method further comprises: acquiring multi-view images of a preset chessboard at different angles and distances by the image acquisition device, thereby determining camera parameters of the image acquisition device.

[0017] According to another aspect of the present disclosure, a 3D head mesh generation method is provided, comprising: acquiring multiple 2D facial images of a target object, the 2D facial images including a frontal face image, a right side face image, and a left side face image, the frontal face image having a circular reference object at the forehead position of the target object; reconstructing a 3D head shape of the target object based on the multiple 2D facial images; determining the head size of the target object based on the frontal face image; and generating a 3D head mesh of the target object having the head size.

[0018] In one embodiment, determining the head size of the target object based on the frontal face image includes: determining the head size of the target object based on the frontal face image according to the above-mentioned head size measurement method.

[0019] In one embodiment, reconstructing the 3D head shape of the target object based on the multiple 2D face images includes:

[0020] The 3D head shape of the target object is reconstructed based on the multiple 2D face images using an encoder based on a 3DMM based on head shape and texture to obtain a reconstructed 3D head shape.

[0021] In one embodiment, the method further comprises: pre-training the head shape and texture based 3DMM based on training data.

[0022] In one embodiment, the pre-training of the 3DMM based on head shape and texture based on training data includes: generating a parameterized head based on the training data using a non-rigid iterative closest point (NICP) algorithm; aligning the parameterized head to a uniform size using a generalized Procter & Gauss analysis (GPA); and extracting principal components (PCs) from all parameterized head meshes using PCA to obtain the 3DMM based on head shape and texture.

[0023] In one embodiment, a 3D head shape of the target object is reconstructed based on the multiple 2D face images using an encoder based on a 3DMM of head shape and texture, and obtaining the reconstructed 3D head shape includes: inputting the multiple 2D face images into an artificial neural network encoder to predict 3D head mesh parameters, the 3D head mesh parameters including 3DMM coefficients, lighting parameters, and camera parameters of the 3D head mesh to minimize a network loss function, the network loss function including: pixel-level loss L img (ξ), the perceptual identity loss L of the reconstructed face image ide (ξ), facial edge loss L edge (ξ) and projection feature point loss L lan (ξ).

[0024] In one embodiment, the method further comprises: extracting perceptual features used in perceptual identity loss calculation using a pre-trained face recognition network.

[0025] According to another aspect of the present disclosure, a 3D head mesh using method is provided, which is applied to a 3D head mesh generated according to the above-mentioned 3D head mesh generating method, comprising: using the 3D head mesh for headwear design.

[0026] In one embodiment, the headgear includes headphones, glasses, a helmet, a mask, or a hat.

[0027] According to yet another aspect of the present disclosure, a head size measurement device is provided, comprising: a facial image acquisition module for acquiring a facial image of a target subject, the facial image having a circular reference object at the forehead position of the target subject; a facial feature point determination module for determining the facial feature points of the target subject; a local area determination module for determining a local facial area including the circular reference object based on the facial feature points; a reference object advance module for extracting the circular reference object from the local facial area; and a head size determination module for determining a diameter of the circular reference object, and determining the head size of the target subject based on the diameter of the circular reference object.

[0028] According to another aspect of the present disclosure, a 3D head mesh generation device is provided, comprising: a facial image acquisition module for acquiring multiple 2D facial images of a target object, the 2D facial images comprising a frontal facial image, a right side facial image, and a left side facial image, the frontal facial image having a circular reference object at the forehead position of the target object; a head shape reconstruction module for reconstructing a 3D head shape of the target object based on the multiple 2D facial images; a head size determination module for determining the head size of the target object based on the frontal facial image; and a head mesh generation module for generating a 3D head mesh having the head size of the target object.

[0029] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-mentioned head size measurement method or 3D head mesh generation method by executing the executable instructions.

[0030] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the above-mentioned head size measurement method or 3D head mesh generation method.

[0031] The head size measurement method and 3D head mesh generation method provided by the embodiments of the present disclosure determine the head size of a target object based on the size of a circular reference object, and have a high accuracy rate in size determination.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0034] FIG1 is a schematic diagram showing the structure of a computer system according to an embodiment of the present disclosure;

[0035] FIG2 shows a flow chart of a method for measuring head size according to an embodiment of the present disclosure;

[0036] FIG3 shows a schematic diagram of a square mark in one embodiment of the present disclosure;

[0037] FIG4 shows a schematic diagram of a circular mark in one embodiment of the present disclosure;

[0038] FIG5A shows a flow chart of a circular reference object extraction method according to an embodiment of the present disclosure;

[0039] FIG5B shows a schematic diagram of a circular reference object measurement channel in one embodiment of the present disclosure;

[0040] FIG6A shows a flowchart of a method for generating a D head mesh in one embodiment of the present disclosure;

[0041] FIG6B shows a main workflow diagram of a method for generating a 3D head mesh according to an embodiment of the present disclosure;

[0042] FIG7 is a schematic diagram of a 3DMM of a human head according to an embodiment of the present disclosure;

[0043] FIG8 is a schematic diagram showing a 3D head shape reconstruction framework in one embodiment of the present disclosure;

[0044] FIG9 illustrates measurement evaluation of different circular reference objects in one embodiment;

[0045] FIG10 shows a comparison of qualitative results of two methods using circle or square markings in one embodiment;

[0046] Figure 11 shows the application scenario of head-related products;

[0047] FIG12 is a schematic diagram of a head size measuring device according to an embodiment of the present disclosure;

[0048] FIG13 shows a schematic diagram of a 3D head mesh generation apparatus according to an embodiment of the present disclosure; and

[0049] FIG14 shows a structural block diagram of a computer device in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0051] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0052] The solution provided by this application uses a circular object as a measurement reference, and based on the head dimensions determined by the circular reference object, a 3D head mesh is reconstructed for application in ergonomic design. To facilitate understanding, several terms involved in this application are first explained below.

[0053] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0054] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0055] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Furthermore, it involves image processing, transforming the images into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. Common biometric recognition technologies include facial recognition and fingerprint recognition.

[0056] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0057] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0058] The solutions provided in the embodiments of this application involve artificial intelligence computer vision, machine learning and other technologies, which are specifically described through the following examples:

[0059] 1 is a schematic diagram showing the structure of a computer system provided in one embodiment of the present disclosure. The system includes: a plurality of terminals 120 and a server (or server cluster) 140.

[0060] The terminal 120 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the terminal 120 may also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0061] The terminal 120 may be installed with an application program for providing and implementing the method disclosed herein.

[0062] The terminal 120 is connected to the server 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0063] Server 140 is a single server, or a combination of multiple servers, a virtualization platform, or a cloud computing service center. Server 140 provides backend services for applications that provide... Optionally, server 140 performs primary computing tasks, while terminal 120 performs secondary computing tasks; alternatively, server 140 performs secondary computing tasks, while terminal 120 performs primary computing tasks; alternatively, terminal 120 and server 140 utilize a distributed computing architecture for collaborative computing.

[0064] In some optional embodiments, the server 140 is used to store information such as the 3D head model established through training.

[0065] Optionally, the application clients installed in different terminals 120 are the same, or the application clients installed on the two terminals 120 are clients of the same type of application on different control system platforms. Based on the different terminal platforms, the specific form of the application client may also be different. For example, the application client may be a mobile phone client, a PC client, or a World Wide Web (Web) client.

[0066] Those skilled in the art will appreciate that the number of the terminals 120 may be greater or less. For example, there may be only one terminal, or there may be dozens, hundreds, or even more terminals. The embodiments of the present application do not limit the number and device types of the terminals.

[0067] Optionally, the system may further include a management device (not shown in FIG1 ), which is connected to the server 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0068] Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0069] Below, each step of the head size measurement and reconstruction method in this exemplary embodiment will be described in more detail with reference to the accompanying drawings and embodiments.

[0070] Figure 2 shows a flowchart of a head size measurement method according to an embodiment of the present disclosure. The method provided by the embodiments of the present disclosure can be executed by any electronic device with computing capabilities, such as terminal 120 and / or server 140 in Figure 1 . In the following examples, server 140 is used as the execution entity.

[0071] As shown in Figure 2, S202, a facial image of a target object is acquired, and the facial image has a circular reference object at the forehead position of the target object. The target object refers to a user. In one embodiment, the facial image uses a frontal facial image of the user. In one embodiment, during the facial image capture process, the user maintains a neutral expression (eyes open, mouth closed) to avoid any expression effects in the 3D head reconstruction. The forehead position may include the middle position of the forehead. In one embodiment, the circular reference object is a printed circular mark, for example, a black circular mark printed on white paper. The circular reference object can also be a reference object similar to a coin, and the color of the circular reference object is not limited to black, and other colors that are easily distinguished from the skin color can also be used.

[0072] S204, determine the facial feature points of the target object from the facial image. In one embodiment, the facial feature points of the target object include eye feature points, eyebrow feature points, or both. Eye feature points may include eye corner feature points and eyeball feature points; eyebrow feature points may be the center of the eyebrow. In one embodiment, facial feature points may include nose feature points,

[0073] S206: Determine a local facial region including the circular reference object based on the facial feature points. Based on the determined facial feature points, the position range of the circular reference object on the target subject's face can be determined, thereby determining a local facial region including the circular reference object. The local facial region can be rectangular or square.

[0074] S208: Extracting a circular reference object from the local facial region. The circular reference object can be extracted from the local facial region using an artificial neural network-based image segmentation network, such as a U-Net or FCN network based on a CNN. The local facial region can be determined relatively accurately using facial feature points, and extracting the circular reference object from the local facial region can improve processing accuracy and speed.

[0075] In one embodiment, circular reference object segmentation based on U-Net includes the following steps: downsampling the facial local area image by Conv+Pooling; then upsampling by Deconv deconvolution, cropping the previous low-level feature map, and performing feature fusion; and then upsampling again. The above process is repeated until a feature map of a predetermined output size is obtained, and finally the output segment map is obtained through softmax. The advantages of the U-Net network include that it only requires a small sample training set for training, is a relatively small segmentation network, and has a simple structure. Through the previous facial local area segmentation, it can better adapt to the situation of U-net and has a high segmentation accuracy.

[0076] S210: Determine the head size of the target object based on the size of the circular reference object. The head size of the target object can be determined based on the extracted size of the circular reference object. The size of the circular reference object can be the diameter, circumference, or area of ​​the circular reference object.

[0077] In the above embodiment, a local facial region is determined using facial feature points, a circular reference object is extracted from the local facial region, and the head size of the target subject is determined based on the dimensions of the circular reference object, resulting in high accuracy. Compared to reference objects of other shapes, circular reference objects are more adaptable to the head region and are less susceptible to distortion caused by deformation, thereby improving measurement accuracy. Placing the circular reference object in the forehead region can reduce deformation of the circular reference object.

[0078] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0079] Before introducing face image capture, we first introduce a way to implement camera parameter measurement.

[0080] Before capturing the facial image, a checkerboard mark template (see, for example, FIG3 ) is printed for camera parameter measurement. For example, a black checkerboard mark can be printed on A4 paper.

[0081] Through these chessboard images, OpenCV is used to calibrate the camera, calculate the camera parameters (i.e., distortion coefficients and intrinsic matrices), and undistort the captured images.

[0082] Camera parameter measurement

[0083] Some cameras introduce significant distortion to the captured images, including radial distortion and tangential distortion. Camera calibration can be performed to determine the distortion coefficients (e.g., five distortion parameters), and then the distortion of the captured images is removed. In addition, the camera's intrinsic parameter matrix K is measured and used for the perspective transformation step of removing image distortion and 3D head reconstruction. The coefficients in this matrix include the focal length (fx, fy) and the optical center (cx, cy).

[0084] In one embodiment, a mobile phone was used as a representative tool to capture multi-view images (e.g., width and height: 3024 × 4025 pixels). A pre-designed chessboard (10 × 7 squares and 9 × 6 inner corners) with a total size of 210 × 297 mm was printed and placed on a flat surface (e.g., a table). All squares in the chessboard were the same size (each square was 26.7 mm), and the size of the squares is an important parameter for determining the actual distance in camera calibration.

[0085] Multiple images of the chessboard are captured at different angles and distances, for example, more than 10 images, to provide a variety of angles with different viewing angles for camera calibration. Using these chessboard images, camera calibration is performed using, for example, OpenCV to calculate camera parameters (i.e., distortion coefficients and intrinsic matrices), and distortion can be removed from the captured images based on the obtained camera parameters.

[0086] The following describes an example of preparing a circular reference object.

[0087] Use a black and white printer with A4 paper to print pre-designed circular markers for head size measurement. The printed template is shown in Figure 4. For example, 54 circular markers can be cut from a single sheet of printed paper. These can be applied to areas of the participant's forehead, such as those with a high degree of flatness.

[0088] Here's a quick guide on how to use a circular reference object.

[0089] 3D head dimensions can be measured from 2D images using a black circle marker with a constant diameter (e.g., 15 mm) as a reference object. The target subject (participant) is asked to place this circle on their forehead, such as the center of the forehead, which is relatively flatter than other facial regions. First, a frontal facial image is captured with the marker attached to the participant's forehead. Then, two face images without the marker are captured from the left and right (e.g., from 10°–50°). During the facial image capture process, the participant is instructed to maintain a neutral expression (eyes open, mouth closed) to avoid any expression effects in the 3D head reconstruction. After the facial image is captured, an image dedistortion method can be applied to it based on the camera parameters determined previously.

[0090] FIG5A shows a flow chart of a circular reference object extraction method according to an embodiment of the present disclosure, and FIG5B shows a schematic diagram of a corresponding circular reference object measurement channel.

[0091] As shown in FIG5A , at S502 , a rectangular region is cropped on the facial image of the target object based on the detected facial feature points.

[0092] In face image preprocessing, a pre-trained artificial neural network (e.g., FAN) is used to crop the face region from the captured face image of the target object to detect facial feature points (e.g., the white dots shown in FIG5B(a)). A facial rectangular region (e.g., the rectangular region shown in FIG5B(a)) is determined based on the facial feature points, thereby extracting a local facial region for cropping. The cropped image is shown in FIG5B(b), for example, with a size of 256×256 pixels.

[0093] S504 , using the pre-trained U-Net to perform accurate circular reference region segmentation to obtain a circular reference object, such as the black circular object shown in FIG5B(c) .

[0094] S506 , using an edge detector, such as a canny edge detector, to filter the edges of the circular reference object to obtain the edges of the circular reference object, as shown in FIG5B( d ).

[0095] S508 , performing ellipse fitting based on the edge of the circular reference object to obtain the diameter of the fitted ellipse, see the circular reference object shown in FIG5B( e ).

[0096] In the above embodiment, the facial local area can be determined relatively accurately by using facial feature points, and extracting the circular reference object from the facial local area can improve the accuracy and speed of processing.

[0097] FIG6A shows a flowchart of a method for generating a 3D head mesh in one embodiment of the present disclosure.

[0098] As shown in FIG6A , S602 , multiple 2D facial images of the target object are acquired, wherein the 2D facial images include a frontal face image, a right side face image, and a left side face image, and the frontal face image has a circular reference object at the forehead position of the target object.

[0099] S604: Reconstruct a 3D head shape of the target object based on multiple 2D face images.

[0100] S606: Determine the head size of the target object based on the frontal face image. Determine the head size of the target object based on the size of a circular reference object in the face verification image.

[0101] S608: Generate a 3D head mesh of the target object with head size.

[0102] In the above embodiment, the head size of the target object is determined based on the circular reference object in the frontal face image, thereby generating a 3D head mesh with accurate head size, which can meet high-precision size requirements and is applicable to applications such as headwear design that have high requirements on head model size.

[0103] FIG6B shows a main workflow diagram of a method for generating a 3D head mesh according to an embodiment of the present disclosure. As shown in FIG6B , the workflow consists of four parts:

[0104] (1) 2D face image capture;

[0105] (2) 3D head shape reconstruction (using the encoder and 3DMM of head shape and texture);

[0106] (3) 3D head size measurement (by measuring the diameter of a circular reference object using U-Net); and

[0107] (4) Application scenarios of reconstructing head mesh.

[0108] The 3D head shape reconstruction is described in detail below.

[0109] In one embodiment, 3D head shape reconstruction is achieved based on 3DMM. 3DMM is the basis for subsequent 3D head shape reconstruction. A large number of head shape 3DMMs are created based on existing training data sets. In addition, a head texture 3DMM model is also created. A specific example of creating a 3DMM model includes the following steps:

[0110] (1) Generate a parameterized head from a training dataset using a non-rigid iterative closest point (NICP) algorithm. The NICP algorithm can adopt, for example, the optimal step-size non-rigid ICP algorithm for surface registration by Amberg B, Romdhani S, Vetter T. The training dataset can include a head shape training dataset and a head texture training dataset.

[0111] (2) aligning the parameterized heads to a uniform size using Generalized Platts Analysis (GPA);

[0112] (3) Principal components (PCs) were extracted from all parameterized head meshes using principal component analysis (PCA) to build a 3DMM model.

[0113] A schematic diagram of the generated 3DMM model of a human head is shown in FIG7 , which includes a head shape and a head texture model.

[0114] FIG8 is a schematic diagram of a 3D head shape reconstruction framework in one embodiment of the present disclosure. As shown in FIG8 , when multiple face images are input, a CNN-based encoder (e.g., ResNet50) is used to predict parameters, including a 3DMM coefficient of a head mesh (including head shape 801 and head texture 802 parameters), as well as a corresponding number of lighting and camera parameters 803, 804. A fitting face image 806 and fitting face landmarks 807, 808 are generated through a Differentiable Renderer 805, and the loss is determined based on the ground-truth face images 809, the ground-truth face edge heatmap 810, and the ground-truth face landmark 811. The parameters ξ of the 3DMM-based shape reconstruction method include α shp ,α tex ,γ light,k ,γ view,k , where k = 1, 2, ..., N I , N I is the number of input images. The main network loss functions that need to be minimized include pixel-level loss L img (ξ), the perceptual identity loss L for rendered face images ide (ξ), facial edge loss L edge (ξ) and the projected landmark loss L lan (ξ); Use a pre-trained face recognition network (Arcface) to extract perceptual features for perceptual identity loss calculation.

[0115] FIG9 illustrates an embodiment of a measurement evaluation of different circular reference objects to evaluate the measurement of different circular reference objects. This measurement method can accurately detect coins of different sizes and colors, such as a 1 Hong Kong dollar (diameter: 25.5 mm), a 0.5 Hong Kong dollar (diameter: 22.5 mm), and a 0.1 Hong Kong dollar (diameter: 17.5 mm). Experiments demonstrate that the disclosed method for measuring coarse-to-fine circular reference objects can accurately and reliably detect reference objects of different sizes and colors, such as coins. The measurement method is accurate, robust, and convenient. This enables the disclosed head reconstruction method to achieve more consistent facial contours and improved reconstruction accuracy.

[0116] Figure 10 shows a comparison of qualitative results of two methods using circular or square markers in one embodiment. As can be seen from Figure 10, the method of the present disclosure using circular markers has higher accuracy than using square markers.

[0117] Figure 11 illustrates a head-related product application scenario, where the top portion represents a solid-color reconstructed head, and the bottom portion represents a textured reconstructed head. The head model generated by the 3D head mesh reconstruction method disclosed herein has higher dimensional accuracy and is more suitable for ergonomic design, such as headwear design. It can be used for personalized customization (personalization), virtual try-on, and size selection for face and head-related products. Examples of head-related product application scenarios include: (a) headphones; (b) sunglasses; (c) VR glasses; (d) masks; and (e) helmets.

[0118] FIG12 is a schematic diagram of a head size measurement device according to an embodiment of the present disclosure. As shown in FIG12 , the head size measurement device includes:

[0119] A facial image acquisition module 1201 is used to acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead position of the target object;

[0120] A facial feature point determination module 1202 is configured to determine facial feature points of the target object;

[0121] a local area determination module, configured to determine a facial local area including the circular reference object based on the facial feature points;

[0122] A reference object advance module 1203 is configured to extract the circular reference object from the facial local area; and

[0123] The head size determination module 1204 is configured to determine the diameter of the circular reference object, and determine the head size of the target object based on the diameter of the circular reference object.

[0124] FIG13 is a schematic diagram of a 3D head mesh generation device according to an embodiment of the present disclosure. As shown in FIG13 , the 3D head mesh generation device includes:

[0125] A facial image acquisition module 1301 is configured to acquire multiple 2D facial images of a target object, wherein the 2D facial images include a frontal face image, a right side face image, and a left side face image, wherein the frontal face image has a circular reference object at the forehead position of the target object;

[0126] A head shape reconstruction module 1302 is configured to reconstruct a 3D head shape of the target object based on the multiple 2D face images;

[0127] A head size determination module 1303 is configured to determine the head size of the target object based on the frontal face image;

[0128] The head mesh generation module 1304 is configured to generate a 3D head mesh of the target object having the head size.

[0129] In the solution of the disclosed embodiment, two high-resolution 3DMMs of head shape and texture are created. The 3DMM-based 3D head shape reconstruction method achieves higher accuracy using a CNN-based encoder. A novel feature point-to-edge loss is introduced in the CNN-based encoder to ensure the overall reconstructed shape accuracy. Experimental results determine the size and cost-effective number of input face images, which is very useful for practical applications.

[0130] In the embodiments of this disclosure, circular reference objects are used instead of square markers. These objects are less sensitive to deformation when attached to the face, making them more stable and easier to measure head dimensions. Furthermore, using coin-shaped objects is common in everyday life. The U-Net-based precise head dimension measurement method is more stable and convenient. The technical solution of this disclosure is more suitable for ergonomic design.

[0131] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0132] The electronic device 1400 according to this embodiment of the present invention is described below with reference to Figure 14. The electronic device 1400 shown in Figure 14 is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0133] As shown in FIG14 , electronic device 1400 is implemented as a general-purpose computing device. Components of electronic device 1400 may include, but are not limited to, the aforementioned at least one processing unit 1410, the aforementioned at least one storage unit 1420, and a bus 1430 connecting various system components (including storage unit 1420 and processing unit 1410).

[0134] The storage unit stores program code, which can be executed by the processing unit 1410, so that the processing unit 1410 performs the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section above. For example, the processing unit 1410 can perform the steps shown in Figures 2 and 6A.

[0135] The storage unit 1420 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 14201 and / or a cache memory unit 14202 , and may further include a read-only memory unit (ROM) 14203 .

[0136] The storage unit 1420 may also include a program / utility 14204 having a set (at least one) of program modules 14205, such program modules 14205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0137] The bus 1430 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0138] Electronic device 1400 can also communicate with one or more external devices 1500 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1500, and / or any device that enables electronic device 1400 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication can occur via input / output (I / O) interface 1450. Furthermore, electronic device 1400 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1460. As shown, network adapter 1460 communicates with other modules of electronic device 1400 via bus 1430. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with electronic device 1500, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0139] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0140] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section of this specification.

[0141] A program product for implementing the above-described method according to an embodiment of the present invention is described. The program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0143] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0144] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0145] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0146] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0147] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0148] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0149] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims. Industrial Applicability

[0150] The present disclosure is applicable to the field of artificial intelligence technology, and is used to solve the problem of inaccurate head size determination during 3D head / face reconstruction in related technologies, thereby achieving the effect of accurately determining the head size of a target object.

Claims

1. A method for measuring head size, comprising: Acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead position of the target object; Determining facial feature points of the target object; Determine a facial local area including the circular reference object based on the facial feature points; Extracting the circular reference object from the local facial area; The size of the target subject's head is determined based on the size of the circular reference object.

2. The method according to claim 1, wherein: Determining the head size of the target object based on the size of the circular reference object comprises: determining a diameter of the circular reference object; The target subject's head size is determined based on the diameter of the circular reference object.

3. The method according to claim 1, wherein: The facial feature points include eye feature points, and the facial partial image including the circular reference object is a rectangular area including the circular reference object.

4. The method according to claim 1, 2 or 3, wherein: The extracting the circular reference object from the local facial area comprises: Using an image segmentation artificial neural network to segment the local facial area to obtain the circular reference object; detecting an edge of the circular reference object using an edge detector; The ellipse fitting method is used to perform ellipse measurement on the edge of the circular reference object, thereby extracting the circular reference object.

5. The method according to claim 4, wherein: The image segmentation artificial neural network is U-Net or a fully convolutional network FCN, and / or the edge detector is a canny edge detector.

6. The method according to claim 1, wherein: Also includes: The facial image of the target object is dedistorted based on camera parameters of an image acquisition device, wherein the camera parameters include distortion coefficients and an intrinsic matrix, and the distortion coefficients include radial distortion coefficients and tangential distortion coefficients.

7. The method according to claim 6, wherein: Also includes: The image acquisition device is used to acquire multi-view images of a preset chessboard at different angles and distances, thereby determining the camera parameters of the image acquisition device.

8. A 3D head mesh generation method, comprising: Acquire multiple 2D face images of the target object, wherein the 2D face images include a front face image, a right face side image, and a left face side image, and the front face image has a circular reference object at the forehead position of the target object; Reconstructing a 3D head shape of the target object based on the multiple 2D face images; Determining the head size of the target object based on the front face image; A 3D head mesh with head dimensions of the target object is generated.

9. The method according to claim 8, wherein: Determining the head size of the target object based on the front face image includes: The head size measurement method according to any one of claims 1 to 7 determines the head size of the target object based on the frontal face image.

10. The method according to claim 8, wherein: The reconstructing the 3D head shape of the target object based on the multiple 2D face images comprises: The encoder of the 3DMM based on the head shape and texture is used to reconstruct the 3D head shape of the target object based on the multiple 2D face images to obtain the reconstructed 3D head shape.

11. The method according to claim 10, wherein: Also includes: The head shape and texture based 3DMM is pre-trained based on training data.

12. The method according to claim 11, wherein: The pre-training of the 3DMM based on head shape and texture based on training data comprises: generating a parameterized head based on the training data using a non-rigid iterative closest point (NICP) algorithm; aligning the parameterized heads to a uniform size using Generic Platts Analysis (GPA); Principal components (PCs) are extracted from all parameterized head meshes using principal component analysis (PCA) to obtain the 3DMM based on head shape and texture.

13. The method according to any one of claims 10 to 12, wherein: Reconstructing the 3D head shape of the target object based on the multiple 2D face images using an encoder of a 3DMM based on head shape and texture, and obtaining the reconstructed 3D head shape includes: The plurality of 2D face images are input into an artificial neural network encoder to predict 3D head mesh parameters, wherein the 3D head mesh parameters include 3DMM coefficients, lighting parameters, and camera parameters of the 3D head mesh, so as to minimize a network loss function, wherein the network loss function includes: a pixel-level loss L img (ξ), the perceptual identity loss L of the reconstructed face image ide (ξ), face edge loss L edge (ξ) and the projection feature point loss L lan (ξ).

14. The method according to claim 13, wherein: Also includes: A pre-trained face recognition network is used to extract perceptual features used in the perceptual identity loss computation.

15. A method for using a 3D head mesh, applied to a 3D head mesh generated by the 3D head mesh generation method according to any one of claims 8 to 14, comprising: The 3D head mesh is used for headwear design.

16. The method according to claim 15, wherein: The headwear includes headphones, glasses, a helmet, a mask, or a hat.

17. A head size measuring device, comprising: A facial image acquisition module, used to acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead position of the target object; A facial feature point determination module, used to determine the facial feature points of the target object; A local area determination module is used to determine the facial local area including the circular reference object based on the facial feature points. area; A reference object advance module, used for extracting the circular reference object from the local facial area; and The head size determination module is used to determine the diameter of the circular reference object, and determine the head size of the target object based on the diameter of the circular reference object.

18. A 3D head mesh generation device, comprising: A face image acquisition module, used to acquire multiple 2D face images of a target object, wherein the 2D face images include a front face image, a right face side image, and a left face side image, and the front face image has a circular reference object at the forehead position of the target object; A head shape reconstruction module, used for reconstructing a 3D head shape of the target object based on the multiple 2D face images; A head size determination module, used to determine the head size of the target object based on the front face image; The head mesh generation module is used to generate a 3D head mesh with head size of the target object.

19. An electronic device comprising: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the head size measurement method described in any one of claims 1 to 7, or the 3D head mesh generation method described in any one of claims 8 to 14 by executing the executable instructions.

20. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the head size measurement method according to any one of claims 1 to 7, or the 3D head mesh generation method according to any one of claims 8 to 14.

21. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the head size measurement method according to any one of claims 1 to 7, or the 3D head mesh generation method according to any one of claims 8 to 14.

Images