Face reconstruction method, system and skin detection device

Abstract

The embodiment of the application provides a face reconstruction method, a system and a skin detection device, wherein the face reconstruction method comprises the following steps: acquiring multi-view face point cloud data and a corresponding face image when a subject is in a first relative position, and multi-view neck point cloud data and a corresponding neck image when the subject is in a second relative position; the first relative position and the second relative position are different relative position relationships between the subject and a collection device; extracting first region point cloud data satisfying a first preset condition from the face point cloud data and the neck point cloud data, and performing registration on the first region point cloud data to obtain face-neck registration point cloud data; extracting second region point cloud data satisfying a second preset condition from the face point cloud data and the neck point cloud data, and performing fusion on the second region point cloud data according to the face-neck registration point cloud data to generate a face-neck fusion model; and performing texture mapping on the face-neck fusion model according to the face image and the neck image. In the registration, fusion and mapping stages of the application, different point cloud extraction rules are adopted, the influence of a support and other non-human body point cloud data is avoided, and the continuity and stability of the face-neck connection are ensured.

Description

Technical Field

[0001] This application relates to the field of skin detection technology, and in particular to a face reconstruction method, system and skin detection device. Background Technology

[0002] In the fields of cosmetic cosmetology, medical aesthetics, and skin health management, the use of color image acquisition and 3D morphological reconstruction technology to digitally model the face and neck region has become an important tool for skin analysis, treatment effect evaluation, and medical auxiliary diagnosis. The face and neck region is characterized by complex shapes, significant curvature variations, and numerous occlusion relationships. To fully acquire the 3D morphology and color texture information of the face and neck, acquisition devices typically need to capture images from multiple perspectives to cover areas prone to scanning blind spots, such as the jawline, under the chin, and the sides of the neck.

[0003] In some existing 3D facial skin detectors, to stabilize the subject's head posture and reduce motion artifacts, a bracket or support structure that contacts a local area of ​​the face is typically used. While this type of bracket helps stabilize the posture during shooting, it introduces physical occlusion that dynamically changes with the shooting angle. Specifically, when the device focuses on the face, the support structure may obscure the area below the jaw and the neck, resulting in missing point clouds or insufficient texture information in these areas. When the device is adjusted to focus on the neck, the same support structure may obscure the lower face or chin area, affecting the integrity and continuity of facial data.

[0004] In this scenario, if a generic multi-view point cloud registration, fusion, and texture mapping workflow is adopted—that is, treating all point clouds acquired from all views as the same type of data and cropping, registering, and fusion them according to a unified region of interest—the following problems are likely to occur: First, the point cloud at the support point cloud or the occlusion boundary may be misjudged as a valid human body surface and participate in registration and fusion, resulting in geometric misalignment in the jaw and neck junction area and unexpected protrusions or depressions on the model surface. Second, during the texture mapping stage, image areas obscured by supports may be incorrectly selected as texture sources due to their small viewing angle and large projection area. This can lead to visual defects on the surface of the 3D model, such as residual support textures, abnormal colors, or broken textures, which seriously affect the model's realism and usability. Summary of the Invention

[0005] The purpose of this application is to address the problem in the prior art where non-human point clouds, such as scaffold point clouds, generated during point cloud registration, fusion, and texture mapping affect the realism and usability of the final model. This application proposes a face reconstruction method, system, and skin detection device that can differentiate point cloud registration, fusion, and texture mapping based on prior information about the shooting target and scaffold occlusion, thereby improving the geometric accuracy and texture realism of face-neck joint modeling.

[0006] In a first aspect, embodiments of this application provide a face reconstruction method, including: The system acquires multi-view facial point cloud data and corresponding facial images of the subject when the subject is in a first relative position, and multi-view neck point cloud data and corresponding neck images of the subject when the subject is in a second relative position; the first relative position and the second relative position are different relative positional relationships between the subject and the acquisition device. Extract a first region point cloud that meets the first preset condition from the facial point cloud data and the neck point cloud data, and register the first region point cloud to obtain the face-neck registered point cloud. Extract a second region point cloud that meets the second preset condition from the facial point cloud data and the neck point cloud data, and fuse the second region point cloud according to the face and neck registration point cloud to generate a face and neck fusion model. Texture mapping is performed on the face-neck fusion model based on the face and neck images.

[0007] In some embodiments, the first preset condition is: the curvature value of the point cloud data is greater than a first threshold, and / or the change in the normal direction of the point cloud data relative to the adjacent region is greater than a second threshold. The point cloud of the first region is registered to obtain the face and neck registered point cloud, including: Register the facial point cloud data of regions that meet the first preset condition to obtain a facial registration point cloud. Register the point cloud regions in the neck point cloud data that meet the first preset condition to obtain the neck registered point cloud. The facial registration point cloud and the neck registration point cloud are registered to obtain the facial-neck registration point cloud.

[0008] In some embodiments, the facial point cloud data that meets the first preset condition includes one or more of the following: upper face point cloud, middle face point cloud, nose point cloud, mouth point cloud, and jaw point cloud. The point cloud regions in the neck point cloud data that meet the first preset condition include one or more of the following: neck region point cloud, subchinial region point cloud, and face-neck connection region point cloud.

[0009] In some embodiments, the facial point cloud data includes a facial point cloud and a first transition point cloud, and the neck point cloud data includes a neck point cloud and a second transition point cloud. The second presupposition condition is: Remove occluded areas from the facial and neck point cloud data. Select point clouds from facial point clouds to construct a facial fusion point cloud; Point clouds are selected from the neck point cloud to form a neck fused point cloud; The first and second transition point clouds are weighted and fused to form the face and neck fused point cloud; The second region point cloud consists of a face fusion point cloud, a neck fusion point cloud, and a face-neck fusion point cloud.

[0010] In some embodiments, the first transition point cloud and the second transition point cloud are weighted and fused to form a face-neck fused point cloud, including: At least one transition boundary shall be provided in the face-neck transition area; The fusion weights of each point cloud in the first and second transition point clouds are calculated respectively. The fusion weights are determined based on at least one of the following: distance from the point to the transition boundary, normal consistency, curvature continuity, depth reliability, view quality, and region label. Based on the fusion weights, the corresponding point clouds in the first and second transition point clouds are weighted and combined to form the face-neck fused point cloud.

[0011] In some embodiments, the transition boundary is generated by fitting the mandibular contour point, chin point, and neck boundary point.

[0012] In some embodiments, texture mapping is performed on a face-neck fusion model based on a face image and a neck image, including: Based on the face-neck boundary relationship on the face-neck fusion model, the units to be mapped on the surface of the face-neck fusion model are divided into face region, neck region or transition region. Adjust the facial viewpoint weights corresponding to the facial images and the neck viewpoint weights corresponding to the neck images based on the segmentation results: Based on the adjusted viewpoint weights, texture sources are selected from the face image and / or neck image for the unit to be mapped, and texture mapping is performed.

[0013] In some embodiments, adjusting the viewpoint weights based on the segmentation results includes: When the unit to be mapped is divided into a facial region, reduce the neck view weight and / or increase the facial view weight. When the unit to be mapped is divided into the neck region, reduce the facial view weight and / or increase the neck view weight; When the unit to be mapped is divided into a transition region, the facial view weight and neck view weight are progressively adjusted according to the distance from the unit to be mapped to the face-neck boundary.

[0014] In some embodiments, the face-neck boundary relationship is a spatial curve or surface fitted by the front start point and the rear end point; The anterior origin point is selected from at least one of the following points on the face-neck fusion model: the point below the lower lip, the mandibular point, or the chin point. The posterior endpoint is selected from at least one of the following points on the face-neck fusion model: the subauricular point, the posterior mandibular point, or the lateral cervical boundary point.

[0015] In some embodiments, before acquiring multi-view facial point cloud data and corresponding facial images of the subject in a first relative position, and multi-view neck point cloud data and corresponding neck images of the subject in a second relative position, the method further includes: Point cloud data collected from multiple perspectives are calibrated to the same coordinate system, and the point cloud data is preprocessed to remove non-human point clouds.

[0016] In some embodiments, different relative positional relationships between the subject and the acquisition device are achieved through at least one of the following methods: Raise or lower the seat the subject is sitting in; Raise or lower the desktop that supports the data acquisition equipment; Raise or lower the acquisition equipment itself.

[0017] Secondly, embodiments of this application provide a face reconstruction system for implementing the face reconstruction method as described above, including: The acquisition module is used to acquire multi-view facial point cloud data and corresponding facial images of the subject when the subject is in a first relative position, and multi-view neck point cloud data and corresponding neck images of the subject when the subject is in a second relative position; the first relative position and the second relative position are different relative positional relationships between the subject and the acquisition device. The registration module is used to extract a first region point cloud that meets the first preset condition from facial point cloud data and neck point cloud data, and to register the first region point cloud to obtain a face-neck registered point cloud. The fusion module is used to extract a second region point cloud that meets the second preset condition from facial point cloud data and neck point cloud data, and fuse the second region point cloud according to the facial and neck registration point cloud to generate a facial and neck fusion model. The texture mapping module is used to perform texture mapping on the face-neck fusion model based on the face image and neck image.

[0018] Thirdly, embodiments of this application provide a skin detection device, including: The shell has a cavity open on one side for the subject's head to be inserted; An image acquisition device, installed inside the cavity, is used to acquire images; The positioning mechanism includes a first positioning element and a second positioning element, both of which are disposed at the opening edge of the cavity. The first positioning element is used to position the subject's head when the housing is in the first relative position, so that the image acquisition device can capture the subject's face. The second positioning component is used to position the subject's head when the housing is in the second relative position, so that the image acquisition device can capture the subject's neck. Among them, the first relative position and the second relative position are different relative positional relationships between the subject and the image acquisition device; The skin detection device also includes a face reconstruction system as described above, which reconstructs faces based on images captured by the image acquisition device.

[0019] In this application, the facial and neck point cloud data are first registered within a group, and then registered between groups, thereby avoiding the participation of point clouds from non-human areas such as stents in the registration calculation. Furthermore, the point cloud data used in the registration stage prioritizes geometrically stable areas with low occlusion risk as the registration basis, fundamentally preventing stent point clouds from being misidentified as valid human surfaces, and ensuring the geometric accuracy and connectivity stability of the face-neck registration.

[0020] In the point cloud fusion stage, by setting transition boundaries in the face-neck transition area and selecting the best transition point cloud, a face-neck fused point cloud is constructed, so that the face and neck point clouds are smoothly connected in the boundary area. While ensuring the integrity of the model surface, it avoids splicing gaps or layering, thus taking into account the registration robustness, face-neck connection continuity and support occlusion avoidance effect.

[0021] In the texture mapping stage, this application divides the unit to be mapped into a face region, a neck region, and a transition region based on the face-neck boundary relationship, and selects, excludes, reduces, or increases the texture viewpoint for different regions. The progressive adjustment of the viewpoint weight in the transition region ensures a smooth transition between the face texture and the neck texture, effectively eliminating visual defects such as residual support texture, color anomalies, or texture breaks on the model surface, and significantly improving the texture realism and usability of the model. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the skin detection device according to an embodiment of this application; Figure 2 This is a schematic diagram of the skin detection device for capturing neck images according to an embodiment of this application; Figure 3 This is a schematic diagram of the skin detection device without the second positioning element according to an embodiment of this application; Figure 4 This is a schematic diagram of the skin detection device for capturing facial images according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the second positioning member according to an embodiment of this application; Figure 6 This is a top view of the second positioning element according to an embodiment of this application; Figure 7 This is a schematic diagram of the structure of the second positioning member that abuts against the center of the surface in an embodiment of this application; Figure 8This is a schematic flowchart of the face reconstruction method according to an embodiment of this application; Figure 9 This is a schematic diagram of a face image of a point cloud extracted during the registration stage of an embodiment of this application. Figure 10 This is a schematic diagram of a face image of a point cloud extracted during the fusion stage of an embodiment of this application. Figure 11 This is a schematic diagram of the face-neck boundary relationship in the texture mapping stage of this application embodiment.

[0023] Explanation of reference numerals in the attached figures: 1. Shell; 11. Cavity; 12. Upper shell; 13. Lower shell; 2. Positioning mechanism; 21. First positioning component; 211. Upper positioning frame; 211a. Column; 211b. Forehead support; 212, Lower positioning frame; 212a, Chin support; 212b, First support; 212c, Second support; 22. Second positioning component; 221. Through hole; 222. Upper support part; 223. Lower support part. Detailed Implementation

[0024] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Although the description of this application will be presented in conjunction with some embodiments, this does not mean that the features of this application are limited to this embodiment. On the contrary, the purpose of describing the application in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of this application. To provide a thorough understanding of this application, many specific details will be included in the following description. This application may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of this application, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0025] It should be noted that in this specification, similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0027] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0028] In the description of this application, it should be understood that "electrical connection" can be understood as physical contact and electrical conduction between components; it can also be understood as the form of connection between different components in a circuit structure through physical lines that can transmit electrical signals, such as copper foil or wires on a printed circuit board (PCB). "Coupled through..." can be understood as electrical conduction through indirect coupling. Indirect coupling can be understood as contactless coupling. Those skilled in the art will understand that coupling refers to a phenomenon where there is close cooperation and mutual influence between the inputs and outputs of two or more circuit elements or electrical networks, and energy is transferred from one side to the other through interaction. To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0029] Please see Figures 1-7 , Figure 1 This is a schematic diagram of the skin detection device according to an embodiment of this application. Figure 2 This is a schematic diagram of the skin detection device for capturing neck images according to an embodiment of this application. Figure 3 This is a schematic diagram of the skin detection device without the second positioning element according to an embodiment of this application. Figure 4 This is a schematic diagram of the skin detection device for capturing facial images according to an embodiment of this application. Figure 5 This is a schematic diagram of the structure of the second positioning member according to an embodiment of this application. Figure 6 This is a top view of the second positioning component according to an embodiment of this application. Figure 7This is a schematic diagram of the structure of the second positioning member that abuts against the center of the surface in an embodiment of this application.

[0030] This application first describes the system composition and hardware structure involved in the applicant's face reconstruction method.

[0031] This application provides a face reconstruction system and a skin detection device, which can be applied to skin detection scenarios in fields such as beauty and medical aesthetics.

[0032] The face reconstruction system includes an acquisition module, a registration module, a fusion module, and a texture mapping module.

[0033] The acquisition module is used to acquire multi-view facial point cloud data and corresponding facial images of the subject when the subject is in a first relative position, and multi-view neck point cloud data and corresponding neck images of the subject when the subject is in a second relative position. The first relative position and the second relative position are different relative positional relationships between the subject and the acquisition device.

[0034] The registration module is used to extract a first region point cloud that meets the first preset conditions from facial point cloud data and neck point cloud data, and to register the first region point cloud to obtain a face-neck registered point cloud.

[0035] The fusion module is used to extract a second region point cloud that meets the second preset conditions from facial point cloud data and neck point cloud data, and to fuse the second region point cloud according to the facial and neck registration point cloud to generate a facial and neck fusion model.

[0036] The texture mapping module is used to perform texture mapping on the face-neck fusion model based on the face image and neck image.

[0037] The modules described above work in sequence and in concert to achieve a complete workflow from data acquisition to model generation and texture mapping. These modules can be integrated into the same processor or deployed in a distributed manner.

[0038] like Figures 1-7 As shown, the skin detection device includes a housing, an image acquisition device, and a positioning mechanism.

[0039] The housing 1 has a cavity 11 open on one side for the subject's head to be inserted.

[0040] In some embodiments, the housing 1 includes an upper housing 12 and a lower housing 13, which together define a cavity 11 for accommodating the face of the subject. The lower housing 13 is an arc-shaped platform structure and is mounted on a mounting bracket, serving as the supporting foundation for the entire housing 1. The upper housing 12 is disposed on top of the lower housing 13 in a manner movable along the arc-shaped extension direction of the lower housing 13.

[0041] An image acquisition device is installed inside cavity 11 for acquiring images.

[0042] In some embodiments, the image acquisition device is disposed on the upper housing 12 and moves along the arc trajectory of the lower housing 13. During the shooting process, whether shooting the face or the neck, image information from multiple angles needs to be acquired. By driving the upper housing to move along the arc direction, the image acquisition device can continuously shoot the subject from different angles, thereby obtaining a complete image sequence covering the required field of view.

[0043] It should be noted that in existing technologies, to achieve multi-angle shooting, multiple cameras are typically required to be placed at different fixed positions, resulting in a large number of cameras, high hardware costs, and a bulky overall structure. In contrast, this embodiment, by placing the image acquisition device on a movable upper housing 12, requires only a single set of cameras to achieve multi-angle shooting along an arc trajectory. This arrangement significantly reduces the number of cameras, lowering hardware costs; it also makes the overall structure of the device more compact and smaller; furthermore, the arc-shaped movement of the upper housing 12 allows for flexible adaptation to different shooting angles of the face and neck, achieving complete and efficient image acquisition.

[0044] Furthermore, to achieve stable movement of the upper shell 12 along the arc extension direction of the lower shell 13, an arc guide structure (not shown in the figure) can be used: The arc-shaped guide structure mainly includes a guide rail and a sliding engagement component. Specifically, an arc-shaped guide rail can be fixedly installed on the edge of the platform structure of the lower housing 13, while a slider that slides with the arc-shaped guide rail is provided at the bottom of the upper housing 12. To ensure smooth movement and guiding accuracy, a scheme of parallel and spaced double guide rails can be adopted.

[0045] Preferably, the arc guide rail can be a V-shaped roller guide rail. Its working principle is to achieve arc motion by the rolling of V-shaped rollers on the arc-shaped V-shaped guide rail surface. The slider can be equipped with a corresponding set of V-shaped rollers that cooperate with the guide rail, replacing sliding friction with rolling friction, thereby effectively reducing friction, lowering drive energy consumption, and extending the service life of the arc guide structure. At the same time, the V-shaped guide rail structure has a self-cleaning function; the rollers can automatically remove foreign objects from the guide rail surface during rolling, which helps maintain long-term operational stability.

[0046] In some embodiments, the upper housing 12 is configured to move to a specific position along the arcuate extension direction of the lower housing 13. Specifically, when used to photograph a subject's face, the upper housing 12 moves to a position where the optical axis of the image acquisition device is deflected 30° to 60° to the left and right relative to the midline of the subject's face. When used to photograph a subject's neck, the upper housing 12 moves to a position where the optical axis of the image acquisition device is deflected 20° to 40° to the left and right relative to the midline of the subject's neck.

[0047] The aforementioned angle range ensures that the image acquisition device fully covers multi-angle image information of the face and neck during its movement along the arc. A 30° to 60° left-right rotation of the optical axis during facial imaging effectively acquires side views of areas such as the cheeks and cheekbones. A 20° to 40° left-right rotation of the optical axis during neck imaging allows for clear acquisition of image data from the front and sides of the neck in a natural position. The multi-angle images obtained from these two imaging sessions meet the data integrity requirements for subsequent 3D morphological reconstruction.

[0048] The positioning mechanism 2 includes a first positioning element 21 and a second positioning element 22, both of which are located at the opening edge of the cavity 11.

[0049] The first positioning member 21 is used to position the subject's head when the housing 1 is in the first relative position, so that the image acquisition device can capture the subject's face.

[0050] In some embodiments, the first positioning member 22 includes an upper positioning frame 211 and a lower positioning frame 212. The upper positioning frame 211 is formed at the upper end of the opening edge of the cavity 11 and is used to abut against the forehead of the subject for positioning. The lower positioning frame 212 is disposed at the lower end of the opening edge of the cavity 11 and is used to abut against the chin of the subject for positioning. Thus, the first positioning member 21 positions the subject's head together through the forehead and chin, achieving stable support during facial photography.

[0051] The second positioning element 22 is used to position the subject's head when the housing 1 is in the second relative position, so that the image acquisition device can capture the subject's neck. The first relative position and the second relative position represent different relative positional relationships between the subject and the image acquisition device.

[0052] In some embodiments, the second positioning element 22 is formed at the upper end of the opening edge of the cavity 11 and is used to abut against the midface of the subject for positioning. The midface includes bony landmark areas such as the forehead, the lower part of the forehead, and the bridge of the nose, which can provide a reliable positioning reference when shooting from the neck, effectively avoiding the impact of head shaking on image quality.

[0053] In some embodiments, the second positioning member 22 protrudes away from the housing 1 relative to the upper positioning frame 211. Based on this protrusion, when the housing 1 is in the second relative position for neck photography, the subject's head often needs to be tilted slightly backward. At this time, the relatively protruding second positioning member 22 can maintain good contact with the center of the subject's face even when the head is tilted back, thereby ensuring the stability and reliability of the positioning effect.

[0054] In some embodiments, the upper positioning frame 211 includes a column 211a and a forehead support 211b disposed at the bottom end of the column 211a. The lower positioning frame 212 includes a chin support 212a, a first support 212b, and a second support 212c. The chin support 212a is fixedly connected to the lower end of the opening edge of the cavity 11; the first support 212b extends from the left side of the chin support 212a to the upper left, and the second support 212c extends from the right side of the chin support 212a to the upper right, with the ends of the first support 212b and the second support 212c connected to each other at the upper end of the opening edge of the cavity 11. The column 211a is fixed at the connection point between the first support 212b and the second support 212c.

[0055] Thus, the chin support 212a forms a stable frame structure through the first support 212b and the second support 212c, which not only provides a reliable installation foundation for the column 211a of the upper positioning frame, but also ensures the structural rigidity of the overall positioning frame during use.

[0056] In some embodiments, the second positioning member 22 is provided with a through hole 221 extending in the vertical direction. The cross-sectional shape of the through hole 221 is adapted to the cross-sectional shape of the column 211a, so that the second positioning member 22 can be detachably fitted onto the column 211a through the through hole 221.

[0057] The second positioning component 22 includes an upper support portion 222 and a lower support portion 223 arranged sequentially in a vertical direction. The upper support portion 222 rests against the forehead area of ​​the subject, and the lower support portion 223 rests against the bridge of the nose area of ​​the subject. By having the upper and lower support portions correspond to different bony landmarks in the mid-face, multi-point positioning of the subject's head is achieved.

[0058] In practical use, this positioning frame has two working modes depending on the shooting requirements: Firstly, when it is necessary to photograph the subject's face, the second positioning element 22 is not fitted onto the column 211a. At this time, the subject's forehead rests against the forehead support 211b, and the chin rests against the chin support 212a, and the positioning for facial photography is completed through the two-point support of the forehead and chin.

[0059] Secondly, when it is necessary to photograph the subject's neck, the second positioning component 22 is fitted onto the column 211a through the through hole 221. At this time, the upper support 222 rests against the subject's forehead, and the lower support 223 rests against the subject's bridge of the nose. Multi-point positioning is achieved using bony landmarks in the midface, ensuring that the head remains stable even in a neck-photographing posture. Furthermore, since the second positioning component 22 is detachable, switching between facial and neck-photographing is convenient and efficient, requiring no additional tools.

[0060] In some embodiments, the upper support portion 222 and the lower support portion 223 are integrally formed.

[0061] The one-piece design can improve the overall structural strength of the second positioning component 22, while simplifying the manufacturing process, reducing production costs, and avoiding the connection loosening problem that may occur with the split structure.

[0062] In one specific embodiment, both the upper support portion 222 and the lower support portion 223 are provided with soft pads at their respective locations for abutting against the subject. The soft pads can be made of soft materials such as silicone or memory foam.

[0063] By adding soft padding, the comfort of the subject's forehead and bridge of the nose can be effectively improved, which is especially suitable for shooting scenarios that require maintaining a certain position for a certain period of time.

[0064] In some embodiments, a mounting bracket is provided at the bottom of the housing 1, which allows the housing 1 to move between a first relative position and a second relative position. The mounting bracket may include a telescopic support rod or a height-adjustable base structure. The first relative position and the second relative position both represent the height of the housing 1 relative to the ground; the first relative position corresponds to a first height, and the second relative position corresponds to a second height, and the first height is different from the second height.

[0065] By adjusting the relative position using the mounting bracket, the operator can switch the housing 1 between a first height and a second height as needed to accommodate the different positioning height requirements for facial and neck photography.

[0066] In another embodiment, the housing 1 itself is not equipped with a mounting bracket, but is placed as an independent device on a height-adjustable electric lifting table. By controlling the lifting of the electric lifting table, the height of the housing 1 relative to the ground can be switched between a first height and a second height, thereby achieving different relative positional relationships between the subject and the image acquisition device.

[0067] With this setup, the shooting height can be flexibly adjusted using external general-purpose equipment without modifying the structure of the housing 1 itself, further simplifying the structural design and manufacturing cost of the device.

[0068] In another embodiment, the housing 1 itself is not provided with a mounting bracket, but is placed on a table or other supporting surface. By adjusting the height of the seat where the subject is sitting, the position of the subject's head relative to the housing 1 is changed, so that when the subject faces the cavity 11, facial photography corresponding to a first relative position and neck photography corresponding to a second relative position can be achieved while the housing 1 remains fixed relative to the ground.

[0069] With this setup, there is no need to install a bracket on the housing 1. The relative height between the subject's head and the housing 1 can be changed by using the lifting function of the external seat, which further simplifies the mechanical structure of the device and reduces manufacturing costs.

[0070] In some implementations, the absolute value of the difference between the first height and the second height is greater than or equal to 10 cm.

[0071] Limiting the height difference to more than 10cm ensures a clear height distinction between the face and neck shooting positions, allowing the image acquisition device to be aimed at the subject's face and neck areas respectively during two shots, avoiding overlapping or misalignment of the shooting areas due to insufficient height difference.

[0072] In some implementations, the absolute value of the difference between the first height and the second height is 15 cm.

[0073] Setting the height difference to 15cm ensures a sufficient and appropriate distance between the face and neck shooting positions. This guarantees that the image acquisition device is accurately aimed at the subject's face and neck areas during the two shots, while also taking into account the subject's comfort and the ease of device operation.

[0074] In this embodiment of the application, the skin detection device further includes a face reconstruction system, which performs face reconstruction based on images captured by the image acquisition device.

[0075] The following describes the complete usage process of the skin detection device.

[0076] First, facial imaging is performed. The housing 1 is adjusted to a first relative position using the mounting bracket. At this point, the first positioning member 21 is usable, and the second positioning member 22 is not mounted on the column 211a. The subject faces the cavity 11, resting their forehead against the forehead support 211b of the upper positioning bracket 211 and their chin against the chin support 212a of the lower positioning bracket 212, thus completing head positioning. Subsequently, the upper housing 12 moves along the arc extension direction of the lower housing 13, causing the camera to capture images of the subject's face from different angles. During this movement, the camera's optical axis deflects 30° to 60° to the left and 60° to the right relative to the midline of the subject's face, for example, 42°, thereby covering multi-angle image information of the face.

[0077] After facial imaging is completed, neck imaging is performed. The housing 1 is adjusted to a second relative position using the mounting bracket, causing a significant displacement of the housing 1 relative to the first height. The operator places the second positioning member 22 onto the column 211a of the upper positioning frame 211 through the through hole 221, at which point the second positioning member 22 protrudes from the upper positioning frame 211. The subject rests their forehead against the upper support 222 of the second positioning member 22 and their nose against the lower support 223, forming a new positioning reference. The mounting bracket can then be adjusted again to ensure the subject's neck is within the camera's field of view. The upper housing 12 moves again along the arc extension direction, causing the camera's optical axis to deflect 20° to 40° (e.g., 25°) to the left and right relative to the midline of the subject's neck, completing multi-angle image acquisition of the region of interest on the neck.

[0078] It should be noted that the shooting order is not limited to the face before the neck. In actual operation, the neck can also be shot first: first, the second positioning piece 22 is fitted onto the column 211a, and the housing 1 is adjusted to the second height; after the neck is shot, the second positioning piece 22 is removed, and the housing 1 is lowered to the first height, and then the face can be shot.

[0079] With the above structure and usage, the skin detection device of this application only needs to set up a single set of cameras, and with the arc movement of the housing 1 and the time-division first positioning member 21 and second positioning member 22, it can realize multi-angle complete imaging of the face and neck. This solution not only significantly reduces the number of cameras, lowers hardware costs, and reduces the overall size, but also simplifies the switching operation between face and neck imaging through the height-adjustable housing 1 and the reliable positioning mechanism 2, ensuring high positioning accuracy, guaranteeing the integrity and consistency of image data, and meeting the needs of three-dimensional morphological reconstruction.

[0080] The following describes the implementation steps of the face reconstruction method of this application.

[0081] Please see Figures 8-11 , Figure 8 This is a flowchart illustrating the face reconstruction method according to an embodiment of this application. Figure 9 This is a schematic diagram of a face image from a point cloud extracted during the registration stage of an embodiment of this application. Figure 10 This is a schematic diagram of a face image from a point cloud extracted during the fusion stage of this application embodiment. Figure 11 This is a schematic diagram of the face-neck boundary relationship in the texture mapping stage of this application embodiment.

[0082] I. Data Acquisition Steps First, the acquisition module in the face reconstruction system acquires multi-view facial point cloud data and corresponding facial images of the subject when the subject is in the first relative position.

[0083] In some implementations, the subject sits in front of the skin detection device, facing the cavity. After head positioning is completed, the acquisition module controls the image acquisition device to capture images of the subject's face from multiple perspectives (covering the left, front, and right sides of the face). Each perspective acquires a set of point cloud data and a corresponding facial image. Then, the relative position between the subject and the image acquisition device is changed to a second relative position, and the acquisition module controls the image acquisition device to capture images of the subject's neck from multiple perspectives (covering the lower left, lower front, and lower right sides of the neck). Each perspective acquires a set of point cloud data and a corresponding neck image. In some implementations, before acquiring multi-view facial point cloud data and neck point cloud data, the method further includes calibrating the point cloud data acquired from multiple perspectives to the same coordinate system and preprocessing the point cloud data to remove non-human point clouds.

[0084] In some implementations, preprocessing the point cloud data includes: after transforming the point cloud data to the same coordinate system, the system performs a step-by-step judgment on each point or triangular facet based on the following conditions: (1) Spatial position judgment: Check whether the left and right coordinates, height coordinates and front and back depth coordinates of the check point fall within the preset reasonable spatial range. For example, for facial shooting point clouds, the top point cloud (such as the background ceiling) and the bottom point cloud (such as the desktop) that exceed the typical height range of the face are judged as non-human point clouds.

[0085] (2) Depth validity judgment: Check whether the depth value of the check point is within the effective depth of field of the acquisition device. Points that are too far away or too close to the depth of field are usually background noise or outliers and should be removed.

[0086] (3) Normal direction determination: Calculate the normal vector of the point and determine whether its normal direction is towards the acquisition device. Points with normals pointing downwards, backwards, or at an angle greater than 90° to the line of sight of the acquisition device usually correspond to the back of the bracket, the inside of the collar, or the background area, and are determined to be non-human point clouds.

[0087] (4) Occlusion risk assessment: Based on the installation position of the bracket in the equipment coordinate system, determine whether the point is located within the bracket occlusion area. Point clouds that may be mapped onto the bracket structure are removed during the preprocessing stage to prevent them from entering the subsequent registration and fusion process.

[0088] (5) Distance judgment with human body key points: Combine the detection results of human face key points (such as nose tip, jaw contour point, chin point, etc.) to determine whether the point cloud is too far from the human body surface. Points that are more than the preset threshold away from human body key points are judged as background noise.

[0089] Points that meet all of the above conditions are retained as valid data and proceed to the next processing step; points that do not meet any of the above conditions are deleted directly during the preprocessing stage.

[0090] It's important to note that the cropping in the preprocessing stage has a different purpose than the cropping in the subsequent registration and fusion stages. The purpose of cropping in the preprocessing stage is to quickly remove obvious non-human noise (such as background, desktop, support, etc.), essentially a coarse screening. The purpose of cropping in the registration stage is to extract geometrically stable corresponding regions. The purpose of cropping in the fusion stage is to retain the effective surfaces required for the final model. These three stages can use different cropping rules and thresholds without interfering with each other.

[0091] In some implementations, the cropping decision in the preprocessing stage can be achieved using a fixed threshold, such as setting a fixed height range [-10cm, 30cm] and a depth range [20cm, 80cm] as the retention interval for human point clouds; or an adaptive threshold can be used, dynamically calculating the cropping boundary based on the statistical distribution of the currently collected data (such as the mean and variance of point cloud density, the mean and standard deviation of depth); or a curve or surface determined by facial key points can be used as the cropping boundary, for example, fitting a spatial curve along the jaw contour points and retaining only the point cloud above the curve (face) or below the curve (neck).

[0092] II. Registration Steps After acquiring facial point cloud data and neck point cloud data, the registration module extracts a first region point cloud that meets the first preset condition from the facial point cloud data and neck point cloud data, and registers the first region point cloud to obtain the face-neck registered point cloud.

[0093] The first preset condition is: the curvature value of the point cloud data is greater than the first threshold, and / or the change in the normal direction of the point cloud data relative to the adjacent region is greater than the second threshold.

[0094] In some implementations, the system calculates the curvature value and normal direction for each point. Curvature describes the degree of curvature of a local surface—the greater the curvature, the more "protruding" or "recessed" the area (e.g., the tip of the nose, the corner of the eye), making it an ideal "geometrically stable region" for registration. Normal direction describes the orientation of a local surface—areas with drastic changes in normal direction (e.g., the side of the bridge of the nose, the angle of the jaw) are also ideal "geometrically stable regions" for registration. If the curvature value of a point is greater than a preset first threshold, and / or the change in its normal direction relative to adjacent regions is greater than a preset second threshold, then that point is determined to belong to the first region point cloud.

[0095] In some implementations, Gaussian curvature is used as a curvature metric, and a first threshold of 0.5 is set. For each point in the point cloud, its Gaussian curvature value is calculated. When the curvature value is greater than 0.5, the point is determined to belong to a stable region with significant geometric features and is included in the first region point cloud.

[0096] Alternatively, the average curvature can be used as a measure of curvature, with the first threshold set to a value between 0.3 and 0.8.

[0097] In some implementations, for each point in the point cloud, the angle between its normal vector and the normal vectors of all points in its neighborhood is calculated, and the maximum or average value is taken. When this value is greater than a second threshold, the point is determined to belong to the first region of the point cloud.

[0098] The second threshold is expressed in angles and ranges from 5° to 30°. Preferably, the second threshold is set to 10°. This value can effectively identify areas of abrupt changes in normal direction, such as the side of the nasal bridge and the jaw angle, while avoiding misidentifying noise points as feature points.

[0099] In some implementations, for facial point cloud data, regions satisfying the first preset condition include one or more of the following: upper face region, mid-face region, nose region, mouth region, and jaw region. These regions have rich geometric features (such as the protrusion of the tip of the nose, the angle of the jaw, etc.), making them suitable as stable correspondence points for registration. For neck point cloud data, regions satisfying the first preset condition include one or more of the following: neck region, submental region, and face-neck junction region.

[0100] In some implementations, the registration module registers the point cloud of the first region to obtain a face-neck registered point cloud. This registration process employs a strategy of group registration and inter-group registration. The first step is intra-group registration of the face: the point cloud regions in the face point cloud data that meet the first preset condition are registered to obtain the face registration point cloud. Specifically, in an implementation that includes three perspectives: left face, front face, and right face, the left face point cloud and the right face point cloud can be registered to the front face point cloud respectively, and then superimposed to obtain the face registration point cloud.

[0101] The second step is intra-neck registration: the point clouds of the neck point cloud data that meet the first preset condition are registered to obtain the neck registration point cloud. Specifically, in an implementation that includes three perspectives: lower left neck, lower neck, and lower right neck, the lower left neck point cloud and the lower right neck point cloud can be registered to the lower neck point cloud respectively, and then superimposed to obtain the neck registration point cloud.

[0102] The third step is inter-group registration: the facial registration point cloud and the neck registration point cloud are registered to obtain the face-neck registration point cloud. Specifically, the neck registration point cloud and the facial registration point cloud are registered in the mandible, submental region, or face-neck connection candidate region.

[0103] In some implementations, the registration of frontal face point clouds and left face point clouds is used as an example for illustration.

[0104] Please refer to it again. Figure 9 . Figure 9Section (a) shows the frontal face region point cloud used for registration with the left face region point cloud. Figure 9 Part (b) is from Figure 9 (a) shows the effective point cloud extracted from the frontal face region point cloud, which removes the right face region point cloud and retains only the left-leaning region point cloud used for registration with the left face. Figure 9 Section (c) shows the left face region point cloud used for registration with the frontal face region point cloud. Figure 9 Part (d) in the text is from Figure 9 (c) shows the effective point cloud extracted from the point cloud of the left face region, which removed point cloud regions such as the ear region and neck region that have low overlap with the point cloud of the front face or are not suitable for establishing a correspondence.

[0105] Both the frontal and left face point clouds include point clouds of the upper face region, middle face region, nose region, mouth region, and jaw region. These regions have rich geometric features (such as the protrusion of the nose tip, the curvature of the eyebrows, and the angle of the jaw), making them suitable as stable correspondence points for registration.

[0106] It should be noted that when registering the frontal face point cloud with the left face point cloud, the region slightly to the left is extracted, while when registering with the right face point cloud, the region slightly to the right is extracted. In other words, the same set of frontal face point clouds is cropped into different subsets depending on the registration object (left face or right face).

[0107] The registration strategy described above differs fundamentally from the common method of registering and fusing all point clouds at once. The common method treats all point clouds as similar inputs, using a unified region of interest, a unified registration target, and a unified overlap processing rule; while this implementation first performs intra-group registration on point clouds with consistent shooting targets and similar occlusion risks, and then performs inter-group registration through the face-neck connection region. This avoids the support area from participating in registration and improves the stability of the face-neck connection.

[0108] In some implementations, registration can be performed in the following ways, depending on the viewing angle configuration of the acquisition device: When the neck view only includes two views, the lower left neck and the lower right neck, the point cloud of the lower left neck and the point cloud of the lower right neck can be registered first to generate the neck registration point cloud, and then the neck registration point cloud and the face registration point cloud can be registered between groups.

[0109] When the neck viewpoint only includes a single view of the lower neck, the point cloud of that lower neck can be directly used as the neck group data and registered with the facial registration point cloud. This method can be used as a backup solution, suitable for scenarios where the acquisition device cannot achieve multi-angle neck shooting.

[0110] In some implementations, the lower left neck point cloud and the left face point cloud can be paired and registered, and the lower right neck point cloud and the right face point cloud can be paired and registered, instead of the aforementioned process of "first registering within a group and then registering between groups".

[0111] In some implementations, multiple candidate registration results can be jointly optimized to obtain the optimal registration parameters. For example, the errors of multiple registration pairs such as "lower left neck - left face", "lower right neck - right face", and "lower frontal neck - frontal face" can be simultaneously included in the optimization objective function, and the optimal transformation parameters can be solved through global optimization to further improve the registration accuracy.

[0112] III. Integration Steps After obtaining the face and neck registration point cloud, the fusion module extracts the second region point cloud that meets the second preset condition from the face point cloud data and the neck point cloud data, and fuses the second region point cloud according to the face and neck registration point cloud to generate a face and neck fusion model.

[0113] In some implementations, the facial point cloud data includes a facial point cloud and a first transition point cloud, while the neck point cloud data includes a neck point cloud and a second transition point cloud. The first transition point cloud refers to the point cloud located in the facial point cloud data within the region connecting the face and neck (i.e., near the jawline); the second transition point cloud refers to the point cloud located in the neck point cloud data within the region connecting the neck and face (i.e., below the chin and upper neck). Both the first and second transition point clouds are point clouds located in the facial-neck transition region. These two transition point clouds are used together to generate the facial-neck fusion point cloud.

[0114] The second preset conditions include: removing occluded area point clouds from the facial point cloud data and neck point cloud data; selecting point clouds from the facial point cloud to form a facial fusion point cloud; selecting point clouds from the neck point cloud to form a neck fusion point cloud; and weightedly fusing the first transition point cloud and the second transition point cloud to form a face-neck fusion point cloud. The second region point cloud consists of the facial fusion point cloud, the neck fusion point cloud, and the face-neck fusion point cloud.

[0115] It's important to note that the clipping objectives differ between the fusion and registration stages. The registration stage aims to extract stable, repeatable, and suitable local regions for establishing correspondences, focusing on stable correspondence points and registration success rate. The fusion stage, on the other hand, aims to retain the effective surfaces required for the final model while removing scaffolding, occlusion, overlap, or low-quality regions, prioritizing the integrity, continuity, and occlusion avoidance of the final surface. Therefore, the system does not directly reuse the point cloud extracted in the registration stage; instead, it regenerates the region point cloud and fusion weights for fusion based on the surface quality of the final model.

[0116] For facial point cloud fusion, point clouds from the upper face, mid-face, nose, mouth, and upper jaw regions are selected. These regions have good image quality during facial photography (positive viewpoint, reliable depth, and no occlusion), making them suitable as the primary data source for the final facial model.

[0117] For the neck fusion point cloud, point clouds from the subchinular region, the submandibular region, and the neck region were selected. These regions had good image quality when photographed on the neck and were suitable as the main data source for the final neck model.

[0118] The face-neck fusion point cloud is composed of point clouds from a first transition point cloud and a second transition point cloud. In some embodiments, at least one transition boundary is set in the face-neck transition region. A fusion weight is calculated for each point cloud in the first and second transition point clouds, determined based on at least one of the following: distance to the transition boundary, normal consistency, curvature continuity, depth reliability, viewpoint quality, and region label. Based on the fusion weights, corresponding point clouds in the first and second transition point clouds are weighted and combined to form the face-neck fusion point cloud.

[0119] Please refer to it again. Figure 10 . Figure 10 Parts (a), (b), and (c) show the extracted point clouds of the right face region, the frontal face region, and the left face region, respectively. None of the extracted point clouds include any point cloud data of the neck region, which are used to construct the facial fusion point cloud. Figure 10 Parts (d), (e), and (f) show the extracted point clouds of the right neck region, the front neck region, and the left neck region, respectively. None of the extracted point clouds include any point clouds of the facial region; these are used to construct the neck fusion point cloud.

[0120] Therefore, the facial fusion point cloud is derived solely from facial imaging data, and the neck fusion point cloud is derived solely from neck imaging data. The two are strictly separated in terms of data source, thereby avoiding the misuse of support occlusion data that may be included in facial imaging for the neck area, or the misuse of support occlusion data that may be included in neck imaging for the facial area during the fusion stage.

[0121] In some implementations, the transition boundary is generated by fitting the jawline contour points, chin point, and neck boundary points. Specifically, the system can identify the subject's jawline contour points, chin point (chin tip), and neck boundary points using a facial key point detection algorithm, and then fit one or more spatial curves or surfaces as the transition boundary. This transition boundary is not of fixed height, but gradually changes as the position moves from the front to the back of the model to conform to the actual head-neck junction shape.

[0122] After determining the transition boundary, the system generates fusion weights for each point within the transition region based on the following factors: (1) Distance from point to transition boundary: The closer to the face side, the higher the weight of the facial point cloud data; the closer to the neck side, the higher the weight of the neck point cloud data.

[0123] (2) Normal consistency: If the normal direction of a point is consistent with the normal direction of its neighboring points, the weight of that point is higher; if the normal direction changes abruptly (which may indicate data noise or scaffold contamination), the weight of that point is reduced.

[0124] (3) Curvature continuity: If the curvature of a point transitions smoothly with the curvature of its neighboring points, the weight of that point is higher; if the curvature changes abruptly (which may cause sharp corners or breaks on the model surface), the weight of that point is reduced.

[0125] (4) Depth reliability: If the depth value of a point is within the effective depth range of the image acquisition device and the signal-to-noise ratio is high, then the weight of the point is high; if the depth value is unreliable (outside the depth range or the signal-to-noise ratio is low), then the weight of the point is reduced.

[0126] (5) Viewpoint quality: If a point is captured by the image acquisition device at a small viewpoint angle (i.e., directly facing it), the weight of that point is higher; if it is captured at a large viewpoint angle (i.e., at an angle), the weight is lower.

[0127] (6) Region label: If a point is determined to be a skin region by the semantic segmentation model, then the point can participate in the fusion; if it is determined to be a support, clothing or background region, then the point is prohibited from participating in the fusion.

[0128] In one specific implementation, for a point P within the face-neck transition region, the system calculates the sub-weights of each dimension based on six factors as follows: Distance weighting: P is closer to the face side, with a face bias of 0.8 and a neck bias of 0.2; Normal consistency weight: The normal of P is consistent with the facial side neighborhood, with a facial side weight of 0.7 and a neck side weight of 0.3; Curvature continuity weight: The curvature of P is more continuous with the neck side, the face side has a weight of 0.3, and the neck side has a weight of 0.7; Depth reliability weight: P is depth reliable when shooting from the neck, with a weight of 0.6 for the neck side; Viewpoint quality weight: P has a better viewpoint when shooting faces, with a face-side weight of 0.6; Region label: P is identified as a skin region, and neither region is downgraded.

[0129] Combining the sub-weights of the above dimensions (assuming all coefficients are 1 / 6), the following calculation is obtained: W_face = (0.8 + 0.7 + 0.3 + 0.4 + 0.6 + 0.5) / 6 ≈ 0.55, W_neck = 1 - 0.55 = 0.45, Wherein, W_face is the weight of the point originating from the first transition point cloud, calculated based on six factors, and W_neck is the weight of the point originating from the second transition point cloud. In this embodiment, W_face = 0.55, W_neck = 0.45, and W_face > W_neck. This result indicates that the point is dominated by facial data in the final fusion result.

[0130] It should be noted that the "fusion weight" here is not used to "select" data from one side of the two sets of transition point clouds, but rather to control the mixing ratio of the data from both sides in the final fusion result.

[0131] The specific values ​​of the sub-weights for each dimension (such as 0.55 and 0.45) and the preset coefficients for each dimension are provided illustratively for the purpose of understanding the technical solution of this application. In practical applications, the values ​​of the sub-weights and preset coefficients for each dimension can be adjusted according to the actual data quality and application requirements, and this application does not impose specific limitations on this. For example, when the quality of facial imaging is significantly better than that of neck imaging, the preset coefficients of each sub-dimensional on the facial side can be appropriately increased to make the fusion result more inclined towards the facial side data; conversely, when the quality of neck imaging is better, the preset coefficients on the neck side should be increased accordingly.

[0132] Furthermore, the second preset condition also includes an occlusion avoidance strategy: when a certain area is easily occluded by the positioning mechanism, nose occlusion, posture changes, or local deformation during a single shot, the shooting data for that area is deleted or its weight is reduced, and it is supplemented by another set of higher-quality data. For example, during facial shooting, the area below the chin may be occluded by the lower positioning frame, so the facial point cloud data for that area is discarded and supplemented by the unoccluded neck point cloud data obtained during neck shooting. Similarly, during neck shooting, the area above the face may be occluded by the second positioning component, so the neck point cloud data for that area is discarded and supplemented by the unoccluded facial point cloud data obtained during facial shooting.

[0133] IV. Texture Mapping Steps After generating the face-neck fusion model, the texture mapping module performs texture mapping on the face-neck fusion model based on the face image and the neck image.

[0134] Specifically, texture mapping includes the following steps: The first step is region segmentation: Based on the face-neck boundary relationship on the face-neck fusion model, the units to be mapped on the surface of the face-neck fusion model (such as triangular faces, vertices or sampling points) are divided into face regions, neck regions or transition regions.

[0135] The face-neck boundary is a spatial curve or surface fitted from the anterior starting point and the posterior ending point. The anterior starting point is selected from at least one of the points below the lower lip, the mandible, or the chin on the face-neck fusion model; the posterior ending point is selected from at least one of the points below the ear, the posterior mandible, or the lateral neck boundary on the face-neck fusion model. This boundary is not a fixed height, but gradually increases as the triangular surface moves from the front to the back of the model, and can be a downward-concave nonlinear curve to conform to the actual head-neck junction shape.

[0136] The selection of the aforementioned front and rear endpoints is based on the following considerations: During the shooting process, the subject's posture often changes. For example, to expose the neck within the shooting range of the equipment, the subject may need to slightly tilt back, forward, or adjust the angle of their head. These postural changes cause varying degrees of stretching or compression of the skin in the facial and neck areas. Points such as the lower lip point, mandible point, chin point, subauricular point, posterior mandibular point, and lateral neck boundary point are all located on bony landmarks, where the skin adheres closely to the bones, resulting in minimal sliding. Even if the subject adjusts their head posture, the spatial position changes corresponding to these points are primarily determined by overall displacement or rotation, rather than by skin stretching.

[0137] Therefore, the face-neck boundary relationship generated based on the aforementioned bony landmark fitting can maintain geometric consistency when the subject's posture changes—that is, the spatial correspondence between these reference points remains stable during shooting, unaffected by skin deformation. Thus, the face-neck boundary relationship can provide a reliable spatial reference for region division in the texture mapping stage, ensuring accurate alignment and natural transition between facial and neck images in transition areas.

[0138] The second step is to adjust the viewpoint weights: After the region is divided, the texture mapping module adjusts the facial viewpoint weights corresponding to the facial images and the neck viewpoint weights corresponding to the neck images according to the division results.

[0139] Specifically: When the unit to be mapped is classified as a facial region, the neck viewpoint weight is reduced and / or the facial viewpoint weight is increased. This is because the facial region has the best viewpoint, resolution, and lighting conditions when the face is captured, resulting in the highest texture quality; while the neck region may be occluded by the positioning mechanism or have a poor shooting angle when captured, so texture sources should be selected from facial images first.

[0140] When the unit to be mapped is classified as a neck region, the facial viewpoint weights are reduced and / or the neck viewpoint weights are increased. This is because the neck region has the best viewpoint and texture quality when the neck is captured; while when the face is captured, this region may be occluded by the positioning mechanism or be in a blind spot (such as below the jaw), so texture sources should be selected preferentially from the neck image.

[0141] When a unit to be mapped is divided into a transition region, the facial and neck viewpoint weights are progressively adjusted based on the distance from the unit to the face-neck boundary. Specifically, units closer to the boundary (leaning towards the face) receive a higher facial viewpoint weight and a lower neck viewpoint weight; units farther from the boundary (leaning towards the neck) receive a higher neck viewpoint weight and a lower facial viewpoint weight. This progressive adjustment allows the viewpoint to gradually transition from the facial texture to the neck texture, reducing abrupt changes and seams.

[0142] The third step is texture source selection: based on the adjusted viewpoint weights, texture sources are selected from the face image and / or neck image for the unit to be mapped and texture mapping is performed.

[0143] Please refer to it again. Figure 11 . Figure 11 Parts (a) and (b) in the figure represent the frontal and side profile images after applying the face-neck boundary relationship, respectively. The dark area represents the area where facial images of the left, front, and right sides are used for mapping, while the light area represents the area where neck images of the lower left, lower right, and lower right sides are used for mapping. The boundary between the dark and light areas is determined by the face-neck boundary relationship.

[0144] Using the texture mapping method described above, texture sources for the facial and neck regions are selected from the corresponding captured images. The resulting textures have no residual textures from non-human structures such as supports, resulting in natural transitions in the textured areas and significantly improving the realism and consistency of the model surface.

[0145] In some implementations, when visibility is insufficient, the quality of multiple viewpoints is similar, or the general score cannot distinguish between good and bad, the system can also employ alternative viewpoint selection rules. For example, for triangular faces in the neck region that are slightly to the left, right, or center, images of the lower left neck, lower right neck, or directly below the neck are preferred, respectively; for triangular faces in the face region that are slightly to the left, right, or center, images of the face in the corresponding direction are preferred, respectively. If the texture coordinates of a candidate viewpoint fall into the positioning mechanism, bracket, background, or other non-human area, then that viewpoint is prohibited from being used as the texture source for that triangular face, or a larger occlusion penalty is imposed.

[0146] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A face reconstruction method, characterized in that, include: Acquire multi-view facial point cloud data and corresponding facial images of the subject when the subject is in a first relative position, and multi-view neck point cloud data and corresponding neck images of the subject when the subject is in a second relative position. The first relative position and the second relative position are different relative positional relationships between the subject and the acquisition device; Extract a first region point cloud that meets the first preset condition from the facial point cloud data and the neck point cloud data, and register the first region point cloud to obtain a face-neck registered point cloud. Extract a second region point cloud that meets the second preset condition from the facial point cloud data and the neck point cloud data, and fuse the second region point cloud according to the facial and neck registration point cloud to generate a facial and neck fusion model. Texture mapping is performed on the face-neck fusion model based on the face image and the neck image.

2. The face reconstruction method as described in claim 1, characterized in that, The first preset condition is: the curvature value of the point cloud data is greater than a first threshold, and / or the change in the normal direction of the point cloud data relative to the adjacent region is greater than a second threshold; The process of registering the point cloud of the first region to obtain a face-neck registered point cloud includes: Register the facial point cloud data of the regions that meet the first preset condition to obtain a facial registration point cloud. Register the point cloud regions in the neck point cloud data that meet the first preset condition to obtain the neck registered point cloud. The facial registration point cloud and the neck registration point cloud are registered to obtain the facial-neck registration point cloud.

3. The face reconstruction method as described in claim 2, characterized in that, The facial point cloud data that meets the first preset condition includes one or more of the following regions: upper face region point cloud, mid-face region point cloud, nose region point cloud, mouth region point cloud, and jaw region point cloud. The point cloud regions in the neck point cloud data that meet the first preset condition include one or more of the following: neck region point cloud, subchinate region point cloud, and face-neck connection region point cloud.

4. The face reconstruction method as described in claim 1, characterized in that, The facial point cloud data includes a facial point cloud and a first transition point cloud, and the neck point cloud data includes a neck point cloud and a second transition point cloud. The second preset condition is: Remove the occluded area point clouds from the facial point cloud data and the neck point cloud data. Select point clouds from the facial point cloud to form a facial fusion point cloud; Point clouds are selected from the aforementioned neck point cloud to form a fused neck point cloud; The first transition point cloud and the second transition point cloud are weighted and fused to form a face-neck fused point cloud; The second region point cloud is composed of the face fusion point cloud, the neck fusion point cloud, and the face-neck fusion point cloud.

5. The face reconstruction method as described in claim 4, characterized in that, The weighted fusion of the first transition point cloud and the second transition point cloud to form a face-neck fused point cloud includes: At least one transition boundary shall be provided in the face-neck transition area; The fusion weights of each point cloud in the first transition point cloud and the second transition point cloud are calculated respectively. The fusion weights are determined based on at least one of the following: distance from the point to the transition boundary, normal consistency, curvature continuity, depth reliability, view quality, and region label. According to the fusion weight, the corresponding point clouds in the first transition point cloud and the second transition point cloud are weighted and combined to form the face and neck fusion point cloud.

6. The face reconstruction method as described in claim 5, characterized in that, The transition boundary is generated by fitting the mandibular contour point, chin point, and neck boundary point.

7. The face reconstruction method as described in claim 1, characterized in that, The step of performing texture mapping on the face-neck fusion model based on the face image and the neck image includes: Based on the face-neck boundary relationship on the face-neck fusion model, the units to be mapped on the surface of the face-neck fusion model are divided into face region, neck region or transition region. Adjust the facial viewpoint weights corresponding to the facial image and the neck viewpoint weights corresponding to the neck image based on the segmentation results: Based on the adjusted viewpoint weights, a texture source is selected from the face image and / or the neck image for the unit to be mapped, and texture mapping is performed.

8. The face reconstruction method as described in claim 7, characterized in that, The adjustment of viewpoint weights based on the segmentation results includes: When the unit to be mapped is divided into a facial region, the neck view weight is reduced and / or the facial view weight is increased. When the unit to be mapped is divided into the neck region, the facial view weight is reduced and / or the neck view weight is increased. When the unit to be mapped is divided into a transition region, the facial view weight and the neck view weight are progressively adjusted according to the distance from the unit to be mapped to the face-neck boundary relationship.

9. The face reconstruction method as described in claim 7, characterized in that, The face-neck boundary relationship is a spatial curve or surface generated by fitting the front starting point and the rear ending point; The anterior starting point is selected from at least one of the following: the point below the lower lip, the mandibular point, or the chin point on the face-neck fusion model. The posterior endpoint is selected from at least one of the subauricular point, the posterior mandibular point, or the lateral cervical boundary point on the face-neck fusion model.

10. The face reconstruction method as described in claim 1, characterized in that, Before acquiring the multi-view facial point cloud data and corresponding facial image of the subject in the first relative position, and the multi-view neck point cloud data and corresponding neck image of the subject in the second relative position, the method further includes: The point cloud data collected from multiple perspectives is calibrated to the same coordinate system, and the point cloud data is preprocessed to remove non-human point clouds.

11. The face reconstruction method as described in claim 1, characterized in that, The different relative positional relationships between the subject and the acquisition device are achieved through at least one of the following methods: Raise or lower the seat where the subject is sitting; Raise or lower the desktop supporting the acquisition device; Raise or lower the acquisition device itself.

12. A face reconstruction system for implementing the face reconstruction method as described in any one of claims 1-11, characterized in that, include: The acquisition module is used to acquire multi-view facial point cloud data and corresponding facial images of the subject when the subject is in a first relative position, and multi-view neck point cloud data and corresponding neck images of the subject when the subject is in a second relative position. The first relative position and the second relative position are different relative positional relationships between the subject and the acquisition device; The registration module is used to extract a first region point cloud that meets a first preset condition from the facial point cloud data and the neck point cloud data, and to register the first region point cloud to obtain a face-neck registered point cloud. The fusion module is used to extract a second region point cloud that meets the second preset condition from the facial point cloud data and the neck point cloud data, and to fuse the second region point cloud according to the facial and neck registration point cloud to generate a facial and neck fusion model. The texture mapping module is used to perform texture mapping on the face-neck fusion model based on the face image and the neck image.

13. A skin detection device, characterized in that, include: The shell has a cavity open on one side for the subject's head to be inserted; An image acquisition device is installed inside the cavity for acquiring images; The positioning mechanism includes a first positioning element and a second positioning element, both of which are disposed at the opening edge of the cavity. The first positioning member is used to position the subject's head when the housing is in the first relative position, so that the image acquisition device can capture the subject's face. The second positioning element is used to position the subject's head when the housing is in the second relative position, so that the image acquisition device can capture the subject's neck. Wherein, the first relative position and the second relative position are different relative positional relationships between the subject and the image acquisition device; The skin detection device also includes the face reconstruction system as described in claim 12, wherein the face reconstruction system performs face reconstruction based on the images captured by the image acquisition device.

Images