Human body scanning data registration method, device, equipment and medium
By acquiring 3D scan point clouds of the entire head, side profile, and inside the ear, extracting ear point clouds, and calculating spatial transformation relationships, the problem of low compatibility in existing technologies is solved, and more efficient head model generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-26
- Publication Date
- 2026-03-31
AI Technical Summary
Existing human body scan data registration algorithms require a relatively large overlap between 3D scan data of different parts, resulting in low compatibility and an inability to effectively handle interference from non-overlapping areas.
By acquiring 3D scan point clouds of the whole head, side face, and inside the ear, the point cloud of the human ear is extracted respectively, and the point cloud inside the ear is used as a reference to calculate the spatial transformation relationship, improve the overlap rate, reduce the interference of non-overlapping areas, and use a deep network model and point cloud registration algorithm to determine the accurate coordinate transformation relationship.
It improves the overlap rate of human ear point clouds, reduces registration interference in non-overlapping areas, avoids error accumulation, and achieves more accurate human head model generation.
Smart Images
Figure CN121767408A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, and more specifically, to a method, apparatus, device, and medium for registering human body scan data. Background Technology
[0002] In the design and simulation of wearable devices, a large number of human head models are needed to support product design. Currently, 3D scan data of the user's head, ears, and other parts are collected manually, and existing registration algorithms are used to register the 3D scan data of different parts, further generating a human head model based on the registration results. However, because non-overlapping areas between 3D scan data of different parts can interfere with the registration results, existing registration algorithms require relatively small non-overlapping areas, i.e., relatively large overlapping areas, which leads to low compatibility of existing registration algorithms. Therefore, a new registration algorithm urgently needs to be proposed. Summary of the Invention
[0003] One objective of this application is to provide a new technical solution for human body scan data registration.
[0004] According to a first aspect of this application, a method for registering human body scan data is provided, comprising: Acquire 3D point cloud data of the whole head, side face, and inner ear; Human ear 3D scanning point clouds are extracted from the full-head 3D scanning point cloud and the side-face 3D scanning point cloud respectively, to obtain the first human ear 3D scanning point cloud in the full-head 3D scanning point cloud and the second human ear 3D scanning point cloud in the side-face 3D scanning point cloud; Based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear, determine the first spatial transformation relationship and the second spatial transformation relationship; Wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud.
[0005] Optionally, determining the first spatial transformation relationship and the second spatial transformation relationship based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear includes: Calculate the normal vectors of the point clouds in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear, respectively; Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined; Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target space transformation relationship is determined; Wherein, when the first target 3D scan point cloud is the first human ear 3D scan point cloud, the target point cloud pair is the first point cloud pair including the first human ear 3D scan point cloud and the second human ear 3D scan point cloud corresponding to the same human ear position, and the target space transformation relationship is the first space transformation relationship; When the first target 3D scan point cloud is an inner ear 3D scan point cloud, the target point cloud pair is a second point cloud pair including the second human ear 3D scan point cloud and the point clouds in the inner ear 3D scan point cloud that correspond to the same human ear position, and the target spatial transformation relationship is the second spatial transformation relationship.
[0006] Optionally, the step of calculating the normal vectors of the point clouds in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first ear, and the 3D scan point cloud of the second ear includes: Calculate the initial normal vector of the target point cloud, wherein the target point cloud is any one of the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear; Calculate the dot product of the initial normal vector of the target point cloud and the viewpoint direction; When the dot product is negative, the normal vector of the target point cloud is determined as the opposite of the initial normal vector; when the dot product is positive, the normal vector of the target point cloud is determined as the initial normal vector.
[0007] Optionally, determining the target space transformation relationship based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud includes: Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial spatial transformation relationship is determined by the random sampling consensus algorithm. The target spatial transformation relationship is determined based on the initial spatial transformation relationship, the second human ear 3D scan point cloud, the first target 3D scan point cloud, and the iterative nearest point algorithm.
[0008] Optionally, determining the target point cloud pair based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud includes: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined through the first deep network model; The first deep network model is used to extract target point cloud pairs belonging to the same ear location with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud.
[0009] Optionally, the step of determining the target point cloud pair based on the three-dimensional coordinates and normal vector of the second ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud through a first deep network model includes: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial point cloud pair and the confidence level of the initial point cloud pair are determined by the first deep network model. Based on the confidence level of the initial point cloud pairs, select target point cloud pairs from the initial point cloud pairs; The first deep network model is used to extract initial point cloud pairs belonging to the same ear position with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud, and output the confidence of the initial point cloud pairs.
[0010] Optionally, the step of extracting the human ear 3D scan point cloud from the full-head 3D scan point cloud and the side-face 3D scan point cloud respectively to obtain the first human ear 3D scan point cloud in the full-head 3D scan point cloud and the second human ear 3D scan point cloud in the side-face 3D scan point cloud includes: Based on the second target 3D scan point cloud and the second deep network model, the target marker 3D scan point cloud corresponding to the second target 3D scan point cloud is determined. The second deep network model is used to perform semantic segmentation on the point cloud belonging to the human ear in the second target 3D scan point cloud to obtain the target marker 3D scan point cloud containing the human ear 3D scan point cloud marker information. Based on the target marker 3D scan point cloud, determine the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud; Specifically, when the second target 3D scan point cloud is a full-head 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the first human ear 3D scan point cloud; when the second target 3D scan point cloud is a side profile 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the second human ear 3D scan point cloud.
[0011] Optionally, determining the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud based on the target marker 3D scan point cloud includes: Based on the target marker 3D point cloud, extract the human ear 3D scan point cloud from the second target 3D scan point cloud to obtain the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud; Based on the second target 3D scan point cloud, adaptive neighborhood expansion and noise filtering are performed on the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud to obtain the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud.
[0012] According to a second aspect of this application, a human body scan data registration device is provided, comprising: The acquisition module is used to acquire full-head 3D scan point clouds, side-face 3D scan point clouds, and inner ear 3D scan point clouds; The extraction module is used to extract the human ear 3D scanning point cloud from the full head 3D scanning point cloud and the side face 3D scanning point cloud respectively, to obtain the first human ear 3D scanning point cloud in the full head 3D scanning point cloud and the second human ear 3D scanning point cloud in the side face 3D scanning point cloud. The registration module is used to determine a first spatial transformation relationship and a second spatial transformation relationship based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear. Wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud.
[0013] According to a third aspect of this application, an electronic device is provided, the electronic device comprising the means as described in the second aspect; Alternatively, the electronic device includes a memory and a processor, the memory for storing computer instructions and the processor for retrieving the computer instructions from the memory to perform the method as described in any one of the first aspects.
[0014] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.
[0015] This application provides a method for registering human body scan data, including: acquiring a full-head 3D scan point cloud, a side-face 3D scan point cloud, and an inner ear 3D scan point cloud; extracting ear 3D scan point clouds from the full-head and side-face 3D scan point clouds respectively, to obtain a first ear 3D scan point cloud from the full-head 3D scan point cloud and a second ear 3D scan point cloud from the side-face 3D scan point cloud; determining a first spatial transformation relationship and a second spatial transformation relationship based on the inner ear 3D scan point cloud, the first ear 3D scan point cloud, and the second ear 3D scan point cloud; wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side-face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side-face 3D scan point cloud. This method is a novel method for registering human body scan data. In this method, on the one hand, human ear 3D scanning point clouds are extracted from the full-head 3D scan point cloud and the side-face 3D scan point cloud, respectively, so that only the first human ear 3D scanning point cloud in the full-head 3D scan point cloud and the second human ear 3D scanning point cloud in the side-face 3D scan point cloud are retained. In this way, the overlap rate of the first human ear 3D scanning point cloud in the full-head 3D scan point cloud and the second human ear 3D scanning point cloud in the side-face 3D scan point cloud is increased, which can reduce the interference of non-overlapping point clouds on registration. On the other hand, using the second human ear 3D scan point cloud as a reference, the first spatial transformation relationship between the full head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud are calculated respectively. In this way, when generating the user's head model, it is only necessary to transform the full head 3D scan point cloud to the corresponding coordinate system of the side face 3D scan point cloud according to the first spatial transformation relationship, and transform the inner ear 3D scan point cloud to the corresponding coordinate system of the side face 3D scan point cloud according to the second spatial transformation relationship, which can avoid error accumulation.
[0016] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.
[0018] Figure 1 This is a flowchart illustrating a human body scan data registration method provided in this application; Figure 2 This is a schematic diagram of the structure of a human body scan data registration device provided in this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation
[0019] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present application.
[0020] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.
[0021] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0022] In all the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.
[0023] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0024] This application provides a method for registering human body scan data, specifically referring to a user's full-head scan data, side-face scan data, and inner ear scan data. Registration refers to finding the optimal spatial transformation relationship so that human body scan data acquired from different viewpoints, times, or conditions can be aligned to the same coordinate system.
[0025] like Figure 1 As shown, the human body scan data registration method provided in this application includes the following steps S1100 to S1300.
[0026] Step S1100: Obtain full-head 3D scan point cloud, side face 3D scan point cloud, and inner ear 3D scan point cloud.
[0027] In one embodiment of this application, a 3D scanner is used to scan the user's head while rotating 360° around the head to obtain a full-head 3D point cloud. Similarly, keeping the ears free of hair or other obstructions, a 3D scanner is used to scan the user's profile from behind the ears toward the nose while rotating to obtain a side-view 3D point cloud.
[0028] In one embodiment of this application, quick-drying adhesive is injected into the user's ear canal. After the quick-drying adhesive has been fixed and shaped, it is removed. A 3D scanner is used to scan the ear canal while rotating 360° around the fixed quick-drying adhesive to obtain a 3D scanning point cloud of the ear.
[0029] In this embodiment, because hair or other objects may obscure the user's ears, the ear point cloud in the full-head 3D scan point cloud is completed using the 3D point cloud of the side profile that exposes the ears. Since the full-head and side profile 3D scan point clouds are obtained from external scanning, they only include the scan point clouds of the outer ear and part of the inner ear, and do not include the complete inner ear scan point cloud. Therefore, an additional 3D scan point cloud of the inner ear is acquired to obtain the complete inner ear scan point cloud.
[0030] Furthermore, the full-head 3D scan point cloud is used as the full-head scan data, the side-face 3D scan point cloud is used as the side-face scan data, and the inner ear 3D scan point cloud is used as the inner ear scan data. It should be noted that the side-face 3D scan point cloud and the inner ear 3D scan point cloud correspond to the same side of the user. Any point cloud from the full-head 3D scan point cloud, side-face 3D scan point cloud, and inner ear 3D scan point cloud is represented using three-dimensional coordinates.
[0031] Step S1200: Extract the human ear 3D scanning point cloud from the full head 3D scan point cloud and the side face 3D scan point cloud respectively, to obtain the first human ear 3D scan point cloud in the full head 3D scan point cloud and the second human ear 3D scan point cloud in the side face 3D scan point cloud.
[0032] In this embodiment, ear 3D scanning point clouds are extracted from the full-head 3D scan point cloud and the side-face 3D scan point cloud, respectively, so that only the first ear 3D scanning point cloud in the full-head 3D scan point cloud and the second ear 3D scanning point cloud in the side-face 3D scan point cloud are retained. This increases the overlap rate between the first ear 3D scanning point cloud in the full-head 3D scan point cloud and the second ear 3D scanning point cloud in the side-face 3D scan point cloud compared to the overlap rate between the full-head 3D scan point cloud and the side-face 3D scan point cloud. Therefore, in the subsequent step S1300, the interference of non-overlapping point clouds on registration can be reduced.
[0033] In one embodiment of this application, the above step S1200 is specifically implemented through the following steps S1210 and S1211.
[0034] S1210, Based on the second target 3D scan point cloud and the second depth network model, determine the target marker 3D scan point cloud corresponding to the second target 3D scan point cloud.
[0035] The second deep network model is used to perform semantic segmentation on the point cloud belonging to the human ear in the second target 3D scan point cloud, so as to obtain the target labeled 3D scan point cloud containing the human ear 3D scan point cloud label information.
[0036] In this embodiment, the second deep network model is specifically a semantic segmentation model, such as a point-based segmentation model, which is pre-trained. Taking the second target 3D scan point cloud as a full-head 3D scan point cloud as an example, the training process of the second deep network model is as follows: obtaining a second training sample set, which includes multiple sets of second training samples, including full-head 3D scan point clouds and corresponding labeled 3D scan point clouds containing human ear 3D scan point cloud labeling information; and training the second deep network model based on the second training sample set.
[0037] Step S1211: Based on the target marker 3D scan point cloud, determine the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud.
[0038] Specifically, when the second target 3D scan point cloud is a full-head 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the first human ear 3D scan point cloud; when the second target 3D scan point cloud is a side profile 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the second human ear 3D scan point cloud.
[0039] In one embodiment of this application, the 3D scanning point cloud marked by the human ear 3D scanning point cloud marking information in the target marker 3D scanning point cloud is determined as the corresponding human ear 3D scanning point cloud in the second target 3D scanning point cloud.
[0040] In another embodiment of this application, since the second deep network model typically determines the corresponding target marker 3D scan point cloud in the second target 3D scan point cloud through point cloud downsampling, the 3D scan point cloud marked by the ear 3D scan point cloud marker information in the target marker 3D scan point cloud is not a complete ear 3D scan point cloud. Therefore, determining the 3D scan point cloud marked by the ear 3D scan point cloud marker information in the target marker 3D scan point cloud as the corresponding ear 3D scan point cloud in the second target 3D scan point cloud cannot extract the complete ear 3D scan point cloud from the second target 3D scan point cloud. Therefore, the complete ear 3D scan point cloud is extracted from the second target 3D scan point cloud through the following steps S1211-1 and S1211-2.
[0041] Step S1211-1: Based on the target marker 3D point cloud, extract the human ear 3D scan point cloud from the second target 3D scan point cloud to obtain the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud.
[0042] In this embodiment, the 3D scanning point cloud marked by the human ear 3D scanning point cloud marking information in the target marker 3D scanning point cloud is determined as the corresponding initial human ear 3D scanning point cloud in the second target 3D scanning point cloud.
[0043] Step S1211-2: Based on the second target 3D scan point cloud, perform adaptive neighborhood expansion and noise filtering processing on the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud to obtain the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud.
[0044] In this embodiment, based on the second target 3D scan point cloud, an adaptive neighborhood expansion is performed on the corresponding initial human ear 3D scan point cloud within the second target 3D scan point cloud. This allows the extraction of a human ear 3D scan point cloud with granularity aligned with the complete human ear 3D scan point cloud from the second target 3D scan point cloud. Furthermore, based on interference filtering, a clean human ear 3D scan point cloud can be extracted from the second target 3D scan point cloud.
[0045] Step S1300: Determine the first spatial transformation relationship and the second spatial transformation relationship based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear.
[0046] Among them, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud.
[0047] For step S1300 above, specifically based on the 3D scan point clouds of the first and second ears, a point cloud registration algorithm is used to determine the coordinate transformation relationship between the 3D scan point clouds of the first and second ears (represented by rotation and translation matrices). This coordinate transformation relationship is understood to be the same as the coordinate transformation relationship between the full-head 3D scan point cloud and the side-face 3D scan point cloud, i.e., the first spatial transformation relationship. Because the overlap rate between the 3D scan point clouds of the first and second ears is high, an accurate first spatial transformation relationship can be obtained based on them. Furthermore, global registration is achieved through local registration, which reduces the amount of data computation.
[0048] Similarly, based on the 3D point cloud scanned from the inner ear and the second ear, a point cloud registration algorithm is used to determine the coordinate transformation relationship between the 3D point cloud scanned from the inner ear to the second ear. This coordinate transformation relationship is essentially the same as the coordinate transformation relationship between the 3D point cloud scanned from the inner ear to the 3D point cloud scanned from the side of the face, i.e., the second spatial transformation relationship. Because the overlap between the 3D point cloud scanned from the inner ear and the second ear is high, an accurate second spatial transformation relationship can be obtained based on these two point clouds. Furthermore, achieving global registration through local registration reduces the amount of data computation.
[0049] In step S1300 above, using the second ear 3D scan point cloud as a reference, the first spatial transformation relationship between the full head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship between the ear 3D scan point cloud and the side face 3D scan point cloud are calculated respectively. In this way, when generating the user's head model, it is only necessary to transform the full head 3D scan point cloud to the coordinate system corresponding to the side face 3D scan point cloud according to the first spatial transformation relationship, and transform the ear 3D scan point cloud to the coordinate system corresponding to the side face 3D scan point cloud according to the second spatial transformation relationship. Compared to traditional techniques that use the full-head 3D scan point cloud as a benchmark and calculate the spatial transformation relationships between the side-face 3D scan point cloud and the full-head 3D scan point cloud separately, this method avoids error accumulation when generating the user's head model. It first transforms the inside-ear 3D scan point cloud to the corresponding coordinate system of the side-face 3D scan point cloud based on the spatial transformation relationship between the side-face 3D scan point cloud and the full-head 3D scan point cloud. Then, based on the spatial transformation relationship between the side-face 3D scan point cloud and the full-head 3D scan point cloud, it transforms the inside-ear 3D scan point cloud in the side-face 3D scan point cloud coordinate system back to the full-head 3D scan point cloud coordinate system.
[0050] This application provides a method for registering human body scan data, including: acquiring a full-head 3D scan point cloud, a side-face 3D scan point cloud, and an inner ear 3D scan point cloud; extracting ear 3D scan point clouds from the full-head and side-face 3D scan point clouds respectively, to obtain a first ear 3D scan point cloud from the full-head 3D scan point cloud and a second ear 3D scan point cloud from the side-face 3D scan point cloud; determining a first spatial transformation relationship and a second spatial transformation relationship based on the inner ear 3D scan point cloud, the first ear 3D scan point cloud, and the second ear 3D scan point cloud; wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side-face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side-face 3D scan point cloud. This method is a novel method for registering human body scan data. In this method, on the one hand, human ear 3D scanning point clouds are extracted from the full-head 3D scan point cloud and the side-face 3D scan point cloud, respectively, so that only the first human ear 3D scanning point cloud in the full-head 3D scan point cloud and the second human ear 3D scanning point cloud in the side-face 3D scan point cloud are retained. In this way, the overlap rate of the first human ear 3D scanning point cloud in the full-head 3D scan point cloud and the second human ear 3D scanning point cloud in the side-face 3D scan point cloud is increased, which can reduce the interference of non-overlapping point clouds on registration. On the other hand, using the second human ear 3D scan point cloud as a reference, the first spatial transformation relationship between the full head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud are calculated respectively. In this way, when generating the user's head model, it is only necessary to transform the full head 3D scan point cloud to the corresponding coordinate system of the side face 3D scan point cloud according to the first spatial transformation relationship, and transform the inner ear 3D scan point cloud to the corresponding coordinate system of the side face 3D scan point cloud according to the second spatial transformation relationship, which can avoid error accumulation.
[0051] In one embodiment of this application, the above step S1300 is specifically implemented by the following steps S1310 to S1312.
[0052] Step S1310: Calculate the normal vectors of the point clouds in the 3D scan point cloud inside the ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear, respectively.
[0053] In one embodiment of this application, for each point cloud in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second, a normal vector of the corresponding point cloud is calculated using a normal vector calculation algorithm based on, for example, Principal Component Analysis (PCA) algorithm, and the calculated normal vector is directly used as the normal vector of the corresponding point cloud.
[0054] In another embodiment of this application, since the full-head 3D scan point cloud, the side-face 3D scan point cloud, and the inner ear 3D scan point cloud are obtained from different perspectives, times, or conditions—that is, the first ear 3D scan point cloud, the second ear 3D scan point cloud, and the inner ear 3D scan point cloud are obtained from different perspectives, times, or conditions—the normal vector directions of the inner ear 3D scan point cloud, the first ear 3D scan point cloud, and the second ear 3D scan point cloud are inconsistent due to different scanning directions. This affects the accuracy of the target point cloud pairs and the target space transformation relationship calculated subsequently based on the normal vectors. Therefore, step S1310 is specifically implemented through the following steps S1310-1 to S1310-3.
[0055] Step S1310-1: Calculate the initial normal vector of the target point cloud.
[0056] The target point cloud is any one of the following: the 3D scan point cloud inside the ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear.
[0057] In one embodiment of this application, for the target point cloud, the normal vector directly calculated using the normal vector calculation algorithm is used as the initial normal vector of the target point cloud.
[0058] Step S1310-2: Calculate the dot product of the initial normal vector of the target point cloud and the viewpoint direction.
[0059] Here, the viewpoint direction refers to the vector direction from the center of the 3D scanner lens to the point on the surface of the user being measured, and is known in advance. Furthermore, step S1310-2 above is achieved through dot product calculation.
[0060] Step S1310-3: When the dot product is negative, the normal vector of the target point cloud is determined as the opposite of the initial normal vector; when the dot product is positive, the normal vector of the target point cloud is determined as the initial normal vector.
[0061] If the dot product of the initial normal vector of the target point cloud calculated in step S1310-2 and the viewpoint direction is negative, it indicates that the initial normal vector of the target point cloud is opposite to the viewpoint direction. In this case, the normal vector of the target point cloud is updated to the negative of the initial normal vector. Thus, the updated normal vector of the target point cloud is in the same direction as the viewpoint direction.
[0062] Correspondingly, if the dot product calculated in step S1310-2 above is positive, it indicates that the initial normal vector of the target point cloud is in the same direction as the viewpoint. In this case, the normal vector of the target point cloud is determined as the initial normal vector.
[0063] Through the above steps S1310-3, the normal vector of the target point cloud is redirected, ensuring that the normal vector of any point cloud in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first ear, and the 3D scan point cloud of the second ear is consistent.
[0064] Step S1311: Determine the target point cloud pair based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud.
[0065] Specifically, when the first target 3D scan point cloud is the first human ear 3D scan point cloud, the target point cloud pair is the first point cloud pair including the first human ear 3D scan point cloud and the second human ear 3D scan point cloud corresponding to the same human ear position, and the target space transformation relationship is the first space transformation relationship; when the first target 3D scan point cloud is the inner ear 3D scan point cloud, the target point cloud pair is the second point cloud pair including the second human ear 3D scan point cloud and the inner ear 3D scan point cloud corresponding to the same human ear position, and the target space transformation relationship is the second space transformation relationship.
[0066] In this embodiment of the application, when the first target 3D scan point cloud is the first human ear 3D scan point cloud, a first point cloud pair is determined based on the three-dimensional coordinates and normal vector of the first human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, which include point clouds corresponding to the same human ear position in the first human ear 3D scan point cloud and the second human ear 3D scan point cloud.
[0067] Furthermore, if the first target 3D scan point cloud is an inner ear 3D scan point cloud, then based on the three-dimensional coordinates and normal vector of the inner ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the second ear 3D scan point cloud, a second point cloud pair is determined, including the second ear 3D scan point cloud and the point cloud in the inner ear 3D scan point cloud corresponding to the same ear position.
[0068] In one embodiment of this application, a first point cloud pair is determined by using a second human ear 3D scan point cloud as the target point cloud and a first human ear 3D scan point cloud as the source point cloud, based on a point cloud matching algorithm such as Nearest Neighbor Search (NNS). Similarly, a second point cloud pair is determined by using a second human ear 3D scan point cloud as the target point cloud and an inner ear 3D scan point cloud as the source point cloud, based on a point cloud matching algorithm such as Nearest Neighbor Search (NNS).
[0069] Step S1312: Determine the target space transformation relationship based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud.
[0070] Specifically, when the first target 3D scan point cloud is the first human ear 3D scan point cloud, the target space transformation relationship is the first space transformation relationship; when the first target 3D scan point cloud is the inner ear 3D scan point cloud, the target space transformation relationship is the second space transformation relationship.
[0071] In one embodiment of this application, step S1312 is specifically implemented through the following steps S1312-1 and S1312-2.
[0072] Step S1312-1: Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial spatial transformation relationship is determined by the random sampling consensus algorithm.
[0073] In this embodiment, when the target point cloud is the first point cloud pair, the three-dimensional coordinates and normal vectors of the first point cloud pair, the second ear 3D scan point cloud, and the first ear 3D scan point cloud are input into a random sampling consensus algorithm. The random sampling consensus algorithm performs coarse registration of the point clouds in the second and first ear 3D scan point clouds, obtaining a rough coordinate transformation relationship between the first and second ear 3D scan point clouds, which serves as the initial spatial transformation relationship between them. Similarly, the initial spatial transformation relationship between the inner ear 3D scan point cloud and the second ear 3D scan point cloud can be obtained.
[0074] Step S1312-2: Determine the target spatial transformation relationship based on the initial spatial transformation relationship, the second human ear 3D scan point cloud, the first target 3D scan point cloud, and the iterative nearest point algorithm.
[0075] In this embodiment, when the target point cloud pair is the first point cloud pair, the initial spatial transformation relationship between the first and second ear 3D scan point clouds, the 3D coordinates and normal vectors of the second and first ear 3D scan point clouds are input into the iterative nearest point algorithm. This algorithm performs precise registration of the point clouds in the second and first ear 3D scan point clouds, obtaining an accurate coordinate transformation relationship between them. This serves as the first spatial transformation relationship between the first and second ear 3D scan point clouds, representing the coordinate transformation relationship between the full-head and side-face 3D scan point clouds. Similarly, a second spatial transformation relationship between the inner ear and second ear 3D scan point clouds, representing the coordinate transformation relationship between the inner ear and side-face 3D scan point clouds, can be obtained.
[0076] In one embodiment of the application, the above step S1311 is specifically implemented by the following step S1311-1.
[0077] Step S1311-1: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined through the first depth network model.
[0078] The first deep network model is used to extract target point cloud pairs belonging to the same ear location with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud.
[0079] In this embodiment, the first deep network model is pre-trained. Taking the first target 3D scan point cloud as the first human ear 3D scan point cloud as an example, the training process of the first deep network model is as follows: A first training sample set is obtained, which includes multiple sets of first training samples. These first training samples include the first human ear sample 3D scan point cloud extracted from the full-head 3D scan point cloud, the second human ear sample 3D scan point cloud extracted from the side-face 3D scan point cloud, and pre-labeled point cloud pairs at the same ear location in the first and second human ear sample 3D scan point clouds that belong to rigid deformation. The first deep network model is then trained based on the first training sample set. The first deep network model is exemplarily a Point Transformer model based on an attention mechanism.
[0080] In one embodiment of this application, the point cloud pairs at the same ear location exhibiting rigid deformation in the 3D scan point cloud of the first ear sample and the 3D scan point cloud of the second ear sample can be manually calibrated. Of course, other calibration methods can also be used, and this application does not limit the specific methods employed.
[0081] Furthermore, the specific implementation of the above step S1311-1 is as follows: input the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud into the first depth network model, and output the target point cloud pair by the first depth network model.
[0082] Since the first deep network model is trained on point cloud pairs at the same ear location with rigid deformation from the pre-labeled 3D scan point clouds of the first and second ear samples, the target point cloud pairs output by the first deep network model are also point cloud pairs at the same ear location with rigid deformation. This avoids interference from non-rigid deformation of the ear when using target point cloud to calculate the target space transformation relationship.
[0083] In one embodiment of this application, the above step S1311-1 is specifically implemented by the following steps S1311-11 and S1311-2.
[0084] Step S1311-11: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial point cloud pair and the confidence level of the initial point cloud pair are determined through the first deep network model.
[0085] The first deep network model is used to extract initial point cloud pairs at the same ear location that belong to rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud, and output the confidence of the initial point cloud pairs.
[0086] In this embodiment, the point cloud pairs extracted by the first deep network model are denoted as initial point cloud pairs. Furthermore, the first deep network model also calculates the confidence level of the extracted initial point cloud pairs.
[0087] Steps S1311-12: Select target point cloud pairs from the initial point cloud pairs based on their confidence levels.
[0088] In one embodiment of this application, the specific implementation of steps S1311-12 is as follows: the initial point cloud pairs are sorted in descending order of confidence level, and the initial point cloud pairs with confidence levels at the top of the preset percentage are taken as target point cloud pairs. The preset percentage is, for example, 50%.
[0089] In one embodiment of this application, the specific implementation of steps S1311-12 is as follows: Initial point cloud pairs with a confidence level greater than a preset confidence level are identified as target point cloud pairs. The preset confidence level is, for example, 50%.
[0090] By using the above steps S1311-12, the initial point cloud pairs with high confidence are used as target point cloud pairs, which can improve the accuracy of the target point cloud pairs used when calculating the target spatial transformation relationship.
[0091] This application also provides a human body scan data registration device 200, such as Figure 2 As shown, it includes: The acquisition module 210 is used to acquire full-head 3D scan point cloud, side face 3D scan point cloud and inner ear 3D scan point cloud; Extraction module 220 is used to extract human ear 3D scanning point cloud from the full head 3D scanning point cloud and the side face 3D scanning point cloud respectively, to obtain the first human ear 3D scanning point cloud in the full head 3D scanning point cloud and the second human ear 3D scanning point cloud in the side face 3D scanning point cloud. The registration module 230 is used to determine a first spatial transformation relationship and a second spatial transformation relationship based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear. Wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud.
[0092] In one embodiment of this application, the registration module 230 is specifically used for: Calculate the normal vectors of the point clouds in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear, respectively; Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined; Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target space transformation relationship is determined; Wherein, when the first target 3D scan point cloud is the first human ear 3D scan point cloud, the target point cloud pair is the first point cloud pair including the first human ear 3D scan point cloud and the second human ear 3D scan point cloud corresponding to the same human ear position, and the target space transformation relationship is the first space transformation relationship; When the first target 3D scan point cloud is an inner ear 3D scan point cloud, the target point cloud pair is a second point cloud pair including the second human ear 3D scan point cloud and the point clouds in the inner ear 3D scan point cloud that correspond to the same human ear position, and the target spatial transformation relationship is the second spatial transformation relationship.
[0093] In one embodiment of this application, the registration module 230 is specifically used for: Calculate the initial normal vector of the target point cloud, wherein the target point cloud is any one of the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear; Calculate the dot product of the initial normal vector of the target point cloud and the viewpoint direction; When the dot product is negative, the normal vector of the target point cloud is determined as the opposite of the initial normal vector; when the dot product is positive, the normal vector of the target point cloud is determined as the initial normal vector.
[0094] In one embodiment of this application, the registration module 230 is specifically used for: The step of determining the target space transformation relationship based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud includes: Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial spatial transformation relationship is determined by the random sampling consensus algorithm. The target spatial transformation relationship is determined based on the initial spatial transformation relationship, the second human ear 3D scan point cloud, the first target 3D scan point cloud, and the iterative nearest point algorithm.
[0095] In one embodiment of this application, the registration module 230 is specifically used for: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined through the first deep network model; The first deep network model is used to extract target point cloud pairs belonging to the same ear location with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud.
[0096] In one embodiment of this application, the registration module 230 is specifically used for: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial point cloud pair and the confidence level of the initial point cloud pair are determined by the first deep network model. Based on the confidence level of the initial point cloud pairs, select target point cloud pairs from the initial point cloud pairs; The first deep network model is used to extract initial point cloud pairs belonging to the same ear position with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud, and output the confidence of the initial point cloud pairs.
[0097] In one embodiment of this application, the extraction module 220 is specifically used for: Based on the second target 3D scan point cloud and the second deep network model, the target marker 3D scan point cloud corresponding to the second target 3D scan point cloud is determined. The second deep network model is used to perform semantic segmentation on the point cloud belonging to the human ear in the second target 3D scan point cloud to obtain the target marker 3D scan point cloud containing the human ear 3D scan point cloud marker information. Based on the target marker 3D scan point cloud, determine the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud; Specifically, when the second target 3D scan point cloud is a full-head 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the first human ear 3D scan point cloud; when the second target 3D scan point cloud is a side profile 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the second human ear 3D scan point cloud.
[0098] In one embodiment of this application, the extraction module 220 is specifically used to: extract the human ear 3D scanning point cloud from the second target 3D scanning point cloud based on the target marker 3D point cloud, and obtain the corresponding initial human ear 3D scanning point cloud in the second target 3D scanning point cloud; Based on the second target 3D scan point cloud, adaptive neighborhood expansion and noise filtering are performed on the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud to obtain the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud.
[0099] This application also provides an electronic device, which includes any of the human body scanning data registration devices 200 provided in the above-described device embodiments; Or, such as Figure 3 As shown, the electronic device 300 includes a memory 310 and a processor 320. The memory 310 is used to store computer instructions, and the processor 320 is used to retrieve the computer instructions from the memory 310 to execute the method as described in any of the above method embodiments.
[0100] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the above-described method embodiments.
[0101] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.
[0102] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
[0103] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.
[0104] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.
[0105] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0106] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.
[0109] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.
Claims
1. A method for registering human body scan data, characterized in that, include: Acquire 3D point cloud data of the whole head, side face, and inner ear; Human ear 3D scanning point clouds are extracted from the full-head 3D scanning point cloud and the side-face 3D scanning point cloud respectively, to obtain the first human ear 3D scanning point cloud in the full-head 3D scanning point cloud and the second human ear 3D scanning point cloud in the side-face 3D scanning point cloud; Based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear, determine the first spatial transformation relationship and the second spatial transformation relationship; Wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud.
2. The method according to claim 1, characterized in that, The step of determining the first spatial transformation relationship and the second spatial transformation relationship based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear includes: Calculate the normal vectors of the point clouds in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear, respectively; Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined; Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target space transformation relationship is determined; Wherein, when the first target 3D scan point cloud is the first human ear 3D scan point cloud, the target point cloud pair is the first point cloud pair including the first human ear 3D scan point cloud and the second human ear 3D scan point cloud corresponding to the same human ear position, and the target space transformation relationship is the first space transformation relationship; When the first target 3D scan point cloud is an inner ear 3D scan point cloud, the target point cloud pair is a second point cloud pair including the second human ear 3D scan point cloud and the point clouds in the inner ear 3D scan point cloud that correspond to the same human ear position, and the target spatial transformation relationship is the second spatial transformation relationship.
3. The method according to claim 2, characterized in that, The calculation of the normal vectors of the point clouds in the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear includes: Calculate the initial normal vector of the target point cloud, wherein the target point cloud is any one of the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear; Calculate the dot product of the initial normal vector of the target point cloud and the viewpoint direction; When the dot product is negative, the normal vector of the target point cloud is determined as the opposite of the initial normal vector; when the dot product is positive, the normal vector of the target point cloud is determined as the initial normal vector.
4. The method according to claim 2, characterized in that, The step of determining the target space transformation relationship based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud includes: Based on the target point cloud pair, the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud, and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial spatial transformation relationship is determined by the random sampling consensus algorithm. The target spatial transformation relationship is determined based on the initial spatial transformation relationship, the second human ear 3D scan point cloud, the first target 3D scan point cloud, and the iterative nearest point algorithm.
5. The method according to claim 2, characterized in that, The step of determining the target point cloud pair based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud includes: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the target point cloud pair is determined through the first deep network model; The first deep network model is used to extract target point cloud pairs belonging to the same ear location with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud.
6. The method according to claim 5, characterized in that, The step of determining the target point cloud pair based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud through a first deep network model includes: Based on the three-dimensional coordinates and normal vector of the second human ear 3D scan point cloud and the three-dimensional coordinates and normal vector of the first target 3D scan point cloud, the initial point cloud pair and the confidence level of the initial point cloud pair are determined by the first deep network model. Based on the confidence level of the initial point cloud pairs, select target point cloud pairs from the initial point cloud pairs; The first deep network model is used to extract initial point cloud pairs belonging to the same ear position with rigid deformation from the second ear 3D scan point cloud and the first target 3D scan point cloud, and output the confidence of the initial point cloud pairs.
7. The method according to claim 1, characterized in that, The step of extracting human ear 3D scanning point clouds from the full-head 3D scan point cloud and the side-face 3D scan point cloud respectively, to obtain the first human ear 3D scanning point cloud in the full-head 3D scan point cloud and the second human ear 3D scanning point cloud in the side-face 3D scan point cloud, includes: Based on the second target 3D scan point cloud and the second deep network model, the target marker 3D scan point cloud corresponding to the second target 3D scan point cloud is determined. The second deep network model is used to perform semantic segmentation on the point cloud belonging to the human ear in the second target 3D scan point cloud to obtain the target marker 3D scan point cloud containing the human ear 3D scan point cloud marker information. Based on the target marker 3D scan point cloud, determine the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud; Specifically, when the second target 3D scan point cloud is a full-head 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the first human ear 3D scan point cloud; when the second target 3D scan point cloud is a side profile 3D scan point cloud, the human ear 3D scan point cloud in the second target 3D scan point cloud is the second human ear 3D scan point cloud.
8. The method according to claim 7, characterized in that, The step of determining the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud based on the target marker 3D scan point cloud includes: Based on the target marker 3D point cloud, extract the human ear 3D scan point cloud from the second target 3D scan point cloud to obtain the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud; Based on the second target 3D scan point cloud, adaptive neighborhood expansion and noise filtering are performed on the corresponding initial human ear 3D scan point cloud in the second target 3D scan point cloud to obtain the corresponding human ear 3D scan point cloud in the second target 3D scan point cloud.
9. A human body scan data registration device, characterized in that, include: The acquisition module is used to acquire full-head 3D scan point clouds, side-face 3D scan point clouds, and inner ear 3D scan point clouds; The extraction module is used to extract the human ear 3D scanning point cloud from the full head 3D scanning point cloud and the side face 3D scanning point cloud respectively, to obtain the first human ear 3D scanning point cloud in the full head 3D scanning point cloud and the second human ear 3D scanning point cloud in the side face 3D scanning point cloud. The registration module is used to determine a first spatial transformation relationship and a second spatial transformation relationship based on the 3D scan point cloud of the inner ear, the 3D scan point cloud of the first human ear, and the 3D scan point cloud of the second human ear. Wherein, the first spatial transformation relationship is the coordinate transformation relationship between the full-head 3D scan point cloud and the side face 3D scan point cloud, and the second spatial transformation relationship is the coordinate transformation relationship between the inner ear 3D scan point cloud and the side face 3D scan point cloud.
10. An electronic device, characterized in that, The electronic device includes the apparatus as described in claim 9; Alternatively, the electronic device includes a memory and a processor, the memory for storing computer instructions, and the processor for retrieving the computer instructions from the memory to perform the method as described in any one of claims 1-8.
11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.