A skull gender identification method based on multi-modal

Abstract

The application discloses a skull gender identification method based on multi-modal, and relates to the field of skull gender identification. The method comprises the following steps: reconstructing a three-dimensional skull surface model based on skull CT scanning data to obtain a three-dimensional skull data set; for each three-dimensional skull surface model, adopting Koebe iteration to realize conformal mapping from a three-dimensional multi-connected region to a plane circular region to obtain a two-dimensional unit circular disc, and respectively performing normal calculation and coloring, vertex coordinate calculation and coloring, curvature calculation and coloring to obtain a normal map, a spatial position map and a curvature map; inputting the normal map, the spatial position map, the curvature map and a gender label corresponding to the three-dimensional skull surface model into a multi-modal skull gender identification network to train a skull gender identification model; and when the skull gender identification model receives a normal map, a spatial position map and a curvature map of a target three-dimensional skull surface model, the skull gender identification model outputs a corresponding gender. The application can improve the accuracy, efficiency and generalization ability of identification.

Description

Technical Field

[0001] This application relates to the field of skull sex determination, and in particular to a multimodal skull sex determination method. Background Technology

[0002] Skull sex determination plays a crucial role in various fields, including anthropology, archaeology, and forensic medicine. Accurately determining the sex of a skull can provide key clues and strong evidence for numerous aspects, such as criminal investigation, identification of remains, and research on human evolution. Early methods of skull sex determination relied primarily on manual measurement and observation, with experts making subjective assessments of skull morphological characteristics. However, these methods were limited by the experience level and judgment standards of the operators, and due to the complexity of skull morphology, they often failed to comprehensively and systematically reveal the intrinsic connections and patterns between different characteristics.

[0003] With the development of computer vision and deep learning technologies, automated gender identification methods based on 3D data have gradually become a research hotspot. These methods significantly reduce the tediousness of traditional manual operations through automated feature extraction and can fully explore potential patterns and complex relationships within the data. Deep learning models, through their vast parameter spaces and multi-layered structures, not only improve the accuracy of gender determination but also enhance the model's robustness to some extent. However, deep learning methods heavily rely on learning from large amounts of data, while craniofacial data acquisition requires specialized CT or 3D scanning equipment, resulting in limited data scale. Therefore, how to incorporate prior knowledge of the physiological differences in male and female skull morphology to reduce reliance on training data remains an unsolved problem for current deep learning-based methods. Furthermore, effectively processing the high-dimensional features of 3D craniofacial data and accurately preserving key gender identification information remains a major challenge in current research. Summary of the Invention

[0004] The purpose of this application is to provide a multimodal skull sex determination method that can improve the accuracy, efficiency and generalization ability of the determination.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] A multimodal skull sex determination method includes the following steps.

[0007] A three-dimensional skull surface model was reconstructed based on skull CT scan data to obtain a three-dimensional skull dataset.

[0008] For each three-dimensional skull surface model in the three-dimensional skull dataset, Koebe iteration is used to achieve conformal mapping from a three-dimensional multi-connected region to a planar circular domain to obtain a two-dimensional unit disk.

[0009] The two-dimensional unit disc is subjected to normal calculation and coloring, vertex coordinate calculation and coloring, and curvature calculation and coloring respectively to obtain a normal map, a spatial position map, and a curvature map.

[0010] A multi-modal skull gender identification network is constructed, which includes a curvature independent main network and a normal-spatial position shared main network arranged in parallel, a cross-modal bidirectional cross attention network connected with the two main networks, and a multi-scale fusion classification network.

[0011] The normal map, the spatial position map, the curvature map and the gender label corresponding to each three-dimensional skull surface model in the three-dimensional skull data set are input into the multi-modal skull gender identification network for training to obtain a skull gender identification model; wherein the skull gender identification model is used to output the corresponding gender when receiving the normal map, the spatial position map and the curvature map of the target three-dimensional skull surface model as input.

[0012] According to the specific embodiments provided in the present application, the following technical effects are disclosed: To solve the problem of high-dimensional feature extraction and analysis difficulty, in the present application, for a three-dimensional skull surface model, Koebe iteration is used to realize conformal mapping from a three-dimensional multi-connected region to a plane circular domain to obtain a two-dimensional unit disc, that is, the three-dimensional skull feature is extracted by using conformal mapping technology. The three-dimensional skull is regarded as a multi-genus open surface, and conformal mapping is used to convert it into a two-dimensional plane feature image, and then normal calculation and coloring, vertex coordinate calculation and coloring, and curvature calculation and coloring are performed respectively. The vertex, normal vector and curvature information of the three-dimensional model are retained and attached to the two-dimensional feature image, which effectively strengthens the key morphological features and reduces the data dimension, simplifies the feature extraction process, reduces the calculation amount, and makes the gender-related features (such as contour and texture) more easily extracted, thereby providing an efficient and representative feature vector for the subsequent model.

[0013] In the present application, a multi-modal skull gender identification network is also constructed, which includes a curvature independent main network and a normal-spatial position shared main network arranged in parallel, a cross-modal bidirectional cross attention network connected with the two main networks, and a multi-scale fusion classification network. Through the construction and training of the above bidirectional guided multi-modal network, the problem of memory consumption in feature processing of high-resolution images can be optimized, and the lack of skull morphological prior knowledge in existing methods can be compensated. Especially the setting of the cross-modal bidirectional cross attention network, after preliminary feature extraction of the two-dimensional skull disc image, the mapped curvature disc is used as morphological prior information to guide the combined feature extraction of the normal map and the curvature map in stages, and the cross attention mechanism is used to realize the final compression of the features, so that the network performs well in the skull gender identification task and has high identification accuracy and generalization ability.

[0014] In practical applications, the trained skull gender identification model outputs the corresponding gender when receiving the normal map, spatial position map and curvature map of the target three-dimensional skull model as input. Compared with the prior art, such application has higher identification efficiency and is more convenient to use. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0016] Figure 1 The flowchart of the skull gender identification method based on multi-modal in an embodiment of the present application.

[0017] Figure 2 The structural diagram of the multi-modal skull gender identification network.

[0018] Figure 3 The structural diagram of the curvature independent backbone network.

[0019] Figure 4 The connection diagram of the first attention module.

[0020] Figure 5 The structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0022] In order to make the purpose, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0023] In an exemplary embodiment, as shown in Figure 1 A skull gender identification method based on multi-modal is provided, which is executed by a computer device, specifically, can be executed by a terminal or a server, or can be executed by a terminal and a server together. In the embodiments of the present application, the method includes the following steps 101 to 105.

[0024] Step 101, reconstruct a three-dimensional skull curved surface model based on the skull CT scan data to obtain a three-dimensional skull data set.

[0025] In practical applications, in order to effectively solve the problem of insufficient sample size commonly existing in current skull identity feature analysis work, multiple sets of skull CT scan data of multiple regions and multiple different hospitals can be collected, such as 1100 sets. Then, the collected multiple sets of skull CT scan data are screened, and finally 420 sets of complete and available samples are selected, including 420 sets of data with clear gender information annotation (223 sets of male and 197 sets of female). All skull CT scan data are saved in DICOM format, and the resolution of each slice scan image is 512*512 pixels, and the number of slices is between 150 and 250. The richness and diversity of these data provide a solid data foundation for subsequent processing, ensuring the wide applicability of research results.

[0026] In order to construct a three-dimensional skull data set with high geometric precision and high quality, the original skull CT scan data is processed systematically, including the following steps:

[0027] (11) For each set of skull CT scan data, a Gaussian filter is used for smoothing, and a Sobel edge detection operator is used to accurately extract the outline of the skull and skin.

[0028] (12) Based on the outline of the skull and skin, a Marching Cubes algorithm is used for three-dimensional reconstruction of the skull and skin to obtain a three-dimensional reconstruction model.

[0029] (13) The three-dimensional reconstruction model is smoothed to obtain an initial three-dimensional skull model; specifically, to further improve the model quality, the three-dimensional reconstruction model may have defects such as holes and cracks, which are repaired and smoothed. Among them, the smoothing process uses algorithms such as Laplace smoothing to fine-tune the surface vertex position of the three-dimensional reconstruction model, reduce local irregularities, eliminate residual noise effects, and ensure that the three-dimensional reconstruction model has higher precision to meet the stringent requirements of subsequent research on model quality.

[0030] (14) Adjust all initial three-dimensional skull models corresponding to the skull CT scan data to a preset coordinate system, and then cut to obtain multiple skull frontal surface views. Specifically, due to the significant differences in posture, position and scale of the actual collected CT scan images, these differences will interfere with subsequent data analysis and research. In order to effectively eliminate such effects, all initial three-dimensional skull models are adjusted to the Frankfurt coordinate system in this step. Then, based on the pre-set anatomical landmark points, the initial three-dimensional skull model is cut to accurately obtain the skull frontal surface view.

[0031] (15) The cranial anterior view surfaces are aligned and standardized as three-dimensional cranial surface models to obtain a three-dimensional cranial dataset. Specifically, all cranial anterior view surfaces are aligned by using a Thin Plate Spline registration algorithm, and standardized processing is performed.

[0032] After the above series of processing procedures, the differences between different three-dimensional cranial models due to different postures are eliminated, and in terms of scale, all cranial samples are consistent in scale due to the unified reference to the Frankfurt coordinate system. Thus, the present application obtains a three-dimensional cranial dataset with sufficient sample size and high geometric accuracy.

[0033] Step 102, for each three-dimensional cranial surface model in the three-dimensional cranial dataset, a Koebe iteration is used to realize conformal mapping from a three-dimensional multi-connected region to a planar circular domain to obtain a two-dimensional unit disc. Specifically, to solve the difficulty of high-dimensional three-dimensional cranial data feature extraction and analysis, this step gives a fully automated mapping method, which regards the cranial anterior view surface as a three-dimensional high-genus open surface, converts it into a two-dimensional planar multi-hole disc through Koebe iteration, and analyzes it in combination with a two-dimensional image feature extraction network.

[0034] In one specific application, step 102 includes the following steps.

[0035] (21) Boundary extraction is performed on the three-dimensional cranial surface model to obtain bilateral orbital rim boundaries, piriform aperture boundaries, bilateral temporal-sphenoidzygomatic arch gap boundaries, bilateral mandibularzygomatic arch gap boundaries, and curved surface outer contour boundaries.

[0036] Specifically, to ensure that Koebe iteration can be successfully and effectively performed, preprocessing of the cranial surface before iteration is an essential step, and the preprocessing mainly includes hole filling and puncture operations. In particular, the supraorbital foramen, infraorbital foramen, mental foramen and other small holes on the cranial surface have less impact on the overall topological structure of the cranial surface and the main features of interest in subsequent analysis. Based on this, a fine hole filling operation can be performed to ignore the impact of these small holes on the whole. This operation is based on a morphological image processing algorithm, which converts the cranial three-dimensional model into a triangular mesh form by meshing the cranial three-dimensional model. For each triangular patch, calculate its area; and determine an area threshold value based on statistical analysis of a large number of samples; when the triangular patches involved in a hole are all smaller than the threshold value, it is determined that it is a fine hole, and filling is performed.

[0037] After the fine hole filling is completed, the space high genus open surface of the skull is precisely divided. From the perspective of medical anatomy and topology, the following parts are divided: bilateral orbital rim boundary, piriform aperture boundary, bilateral temporal-sphenoid-zygomatic arch gap boundary, bilateral mandibular-zygomatic arch gap boundary, and curved surface outer contour boundary, a total of eight boundaries.

[0038] In terms of data, the front view surface of the skull in this application is represented in the form of a triangular mesh, and the front surface of the skull is denoted as , the vertex set of all triangular facets is denoted as: , for each vertex , the normal vector in the three-dimensional space is: , the edge set is denoted as: , the triangular facet set is denoted as: , and each triangular facet can be specifically represented as , where .

[0039] The bilateral orbital rim boundary is a continuous curve around the eye socket, which defines the spatial range of the eye socket, and can be defined by determining the coordinates of a series of feature points around the eye socket, and there are left and right two. For any one, the feature point set around the eye socket is denoted as , and the coordinates of each point in the three-dimensional space are denoted as , . The bilateral orbital rim boundary can be represented as the ordered connection of these feature points, that is: .

[0040] The piriform aperture boundary is the edge of the piriform bone opening where the nostrils are located, and is one of the important anatomical landmarks of the face. The feature point set around the piriform aperture is denoted as , and the coordinates are denoted as , . The piriform aperture boundary can be represented as: .

[0041] The bilateral temporal-sphenoid-zygomatic arch gap boundary is located in the gap between the temporal bone, sphenoid bone and zygomatic arch, and its shape and position are of great significance for studying the lateral structure of the skull. There are two of them, and the feature point sets around the bilateral temporal-sphenoid-zygomatic arch gap are denoted as and , and the coordinates are denoted as and , and . Then the bilateral temporal-sphenoid-zygomatic arch gap boundary can be represented as: . Wherein, , .

[0042] The bilateral mandibular-zygomatic arch gap boundary is located at the gap between the mandible and the zygomatic arch, and is closely related to the mastication function and appearance of the face. There are two of them, and the feature point set around the bilateral mandibular-zygomatic arch gap is respectively and , the coordinates are respectively and , and . The bilateral mandibular-zygomatic arch gap boundary can be expressed as: ; wherein, , .

[0043] The curved surface outer contour boundary defines the overall shape range of the frontal view of the skull. Let the feature point set around the outer contour be , and the coordinates be , . The outer contour boundary can be expressed as: .

[0044] (22) Seven holes are determined according to the bilateral orbital rim boundary, the piriform aperture boundary, the bilateral temporal-sphenoid-zygomatic arch gap boundary, and the bilateral mandibular-zygomatic arch gap boundary.

[0045] (23) Select any one of the seven holes as the current hole, and fill all the holes except the current hole to obtain a topological ring surface. In a specific application, this step includes the following steps.

[0046] 1) Select any hole other than the current hole as the to-be-filled hole.

[0047] 2) The center point of the to-be-filled hole is determined by using a centroid calculation-based method; let the vertex set on the hole boundary be , and each vertex has coordinates in the three-dimensional space. The coordinates of the center point are calculated by the following formula:

[0048] ; ; .

[0049] 3) Connect the center point to each vertex of the to-be-filled hole in turn to obtain a set of line segments.

[0050] 4) According to the line segment set and the boundary of the hole to be filled, a new triangular patch is determined in combination with the constraint of the triangle inequality. The line segment in the line segment set and the edge on the boundary of the hole to be filled jointly constitute a new triangular patch. In order to ensure that the newly generated triangular patch meets the geometric constraint condition, the triangle inequality (the sum of two edges is greater than the third edge) needs to be met. In this way, the effective filling of the hole is realized.

[0051] 5) The new triangular patch is used to fill the hole to be filled.

[0052] 6) Until all holes except the current hole are filled to obtain a topological torus.

[0053] Based on the above automatic filling operation, not only the complex hole structure on the surface of the skull can be efficiently processed, but also the overall topological features of the skull can be accurately preserved.

[0054] (24) A conformal mapping of the topological torus to a standard planar circular domain is calculated by using a dynamic Yamabe flow to map the current hole as a circular hole. Specifically, for the calculation process in one Koebe iteration of one current hole, the following steps are included: 1) for the topological torus, the cotangent edge weight is calculated. When all edge weights are non-negative, the triangular partition at this time is Delaunay. This triangular partition can guarantee good geometric properties in subsequent calculations; 2) the conformal factor is initialized, and the initial conformal factor is 0; 3) the dynamic Yamabe flow iteration is solved, the current Gaussian curvature is calculated, the Yamabe energy gradient is calculated, the Hessian matrix is calculated, the linear system is solved to update the conformal factor, and the inner boundary is converged to the standard circle until the iteration is ended; 4) the conformal mapping is applied; 5) the updated result surface is obtained.

[0055] wherein the function formula of the dynamic Yamabe flow is:

[0056] .

[0057] wherein, is a discrete conformal factor, which is used to determine the deformation degree of the surface in the conformal transformation; V is a triangular patch vertex set of the three-dimensional skull surface model, is a real number set; is the target curvature of the i-th triangular patch vertex, which refers to the vertex curvature value expected to be reached; is the current discrete Gaussian curvature of the i-th triangular patch vertex at time The formula describes the rate of change of the discrete conformal factor over time, and by continuously adjusting the discrete conformal factor, the curvature of the surface is close to the target curvature.

[0058] (25) until seven hole mapping is completed, get with the three-dimensional skull surface model conformal equivalence of a plurality of circular hole two-dimensional unit disc.

[0059] In each round of iteration, the application selects a hole, fills the remaining holes to obtain a new topological torus, repeats the calculation process based on the dynamic Yamabe flow until the convergence condition is met, maps the hole processed in this round to a circle, and until all holes are mapped to a standard circle. The method based on the discrete homogenization theory is introduced in the above processing process, and the conversion from a three-dimensional multi-connected skull surface to a two-dimensional unit disc is realized by means of Koebe iteration, which can effectively solve the problem of high computational complexity caused by the complex topological structure and irregular geometry of the skull. The data of the obtained two-dimensional unit disc can be represented as: the vertex set is , and the triangular facet set is For each vertex , its normal vector in three-dimensional space is still the original .

[0060] Step 103, normal calculation and coloring, vertex coordinate calculation and coloring, curvature calculation and coloring are performed on the two-dimensional unit disc respectively to obtain normal map, spatial position map and curvature map; specifically, after successfully obtaining the two-dimensional unit multi-hole disc conformally equivalent to the three-dimensional skull surface model, in order to more intuitively display the feature information of the skull surface, further generate the skull geometric feature image, that is, use the image of the skull conformal mapping to the plane domain to store the coordinates, normal vector, curvature and other information of the vertices of the original three-dimensional space surface.

[0061] In one specific application, normal calculation and coloring, vertex coordinate calculation and coloring are performed on the two-dimensional unit disc respectively to obtain normal map and spatial position map, including the following steps.

[0062] 31) Based on the two-dimensional unit disc, determine the triangular facet set of the corresponding three-dimensional skull surface model.

[0063] 32) For each triangular facet in the triangular facet set, based on any vertex, calculate two edge vectors, and then perform cross multiplication operation on the two edge vectors to obtain a normal vector; for each triangular facet , starting from the vertex , calculate two edge vectors and ; the direction of the normal vector is obtained through cross multiplication operation, that is , the vector is perpendicular to the plane formed by the edge vectors, and the direction follows the right-hand rule (if the vertices are arranged in counterclockwise order, the normal vector points to the outside).

[0064] 33) For each triangle patch in the set of triangle patches, calculate the patch barycenter coordinates by arithmetic mean of the three vertex coordinates; for each triangle patch , the corresponding vertex of the original three-dimensional discrete surface after mapping contains three coordinate components x, y, z. For the coordinates of the triangle patch center point (barycenter) , the arithmetic mean of the three vertex coordinates can be calculated:

[0065] .

[0066] 34) Normalize the normal vector and the patch barycenter coordinates respectively; to ensure that the normal vector is of unit length, normalization is required, that is, , represents the length of the normal vector. The normalization of the patch barycenter coordinates is the same.

[0067] 35) Use interval conversion to map the normalized normal vector to RGB color values and assign it to the two-dimensional unit disc to obtain the normal map; the normalized normal vector can be directly mapped to RGB color values, specifically by converting each component from to interval, that is: , , . According to the needs, the floating-point values can be quantized to 8-bit unsigned integers, that is , , , and finally the color of each patch can be represented as a four-tuple . In this way, a corresponding relationship between the normal vector and the color is established, such as the positive X-axis direction corresponding to pure red , the positive Y-axis corresponding to pure green , and the positive Z-axis corresponding to pure blue .

[0068] 36) Use interval conversion to map the normalized patch barycenter coordinates to RGB color values and assign it to the two-dimensional unit disc to obtain the spatial position map. The normalization of the patch barycenter coordinates is the same.

[0069] In addition, curvature calculation and coloring are performed on the two-dimensional unit disc to obtain a curvature map, including the following steps.

[0070] 41) Based on the two-dimensional unit disc, determine the corresponding set of triangle patch vertices of the three-dimensional skull surface model.

[0071] 42) For each vertex in the set of triangle patch vertices, determine the geodesic neighborhood; specifically, the neighborhood is defined as the geodesic neighborhood centered at the ith vertex with geodesic distance as the radius. The grid is traversed by breadth-first search, starting from the ith vertex, accumulating the length of the shortest path (i.e. the geodesic distance) between vertices, and stopping when the path length exceeds . The vertices and edges obtained by the traversal constitute the neighborhood .

[0072] 43) Based on the vertices within the geodesic neighborhood, the neighborhood area of the geodesic neighborhood is calculated; wherein the neighborhood area is the sum of the areas of all triangles within the neighborhood, and the calculation formula is:

[0073] .

[0074] wherein, can be calculated by the vector cross product from the coordinates of the three vertices of the triangle , , .

[0075] 44) Based on the edges within the geodesic neighborhood, the geometric properties of each edge are calculated; specifically, the edge is determined by the endpoints and , the edge vector is defined as: ; the unit vector is: ; is the Euclidean length of the edge . represents the effective length of the edge truncated by the neighborhood . If the edge is completely contained in the neighborhood , then , if partially contained, it is simplified to (approximate processing).

[0076] Let the normal vectors of the two triangles on either side of the edge be and . First, calculate the cosine value of the angle between the normal vectors , which is limited to to avoid numerical problems, obtaining , where clip is a clipping function that limits the dot product result to [-1, 1] to meet the value requirements of the cosine function; then determine the sign by the right-hand rule, where is the cross product of the two normal vectors, is the edge vector. If , it indicates that the edge​ is located in the convex region; , which represents the signed angle between the two vectors , where is in the range of , the sign of determines the sign of .

[0077] 45) Based on the neighborhood area of the geodesic neighborhood and the geometric properties of each edge, a curvature matrix is calculated; specifically, for each edge , the outer product matrix is calculated and multiplied by to obtain the contribution of each edge to the matrix, and then the contributions of all edges are accumulated to the matrix , that is: ; the matrix is divided by the neighborhood area to obtain the curvature matrix . This matrix characterizes the local bending properties of the surface at the vertex through the changes in the normal vectors (represented by ) and the directions (represented by ) of the edges in the neighborhood. The curvature matrix of the vertex i is:

[0078] .

[0079] where, , represents the angle between the normal vectors of the triangular patches on both sides of the edge ; and is the outer product matrix (cross product matrix) of the unit vectors , with elements ; the unit vectors are calculated according to the edge e.

[0080] 46) The eigenvalue decomposition of the curvature matrix is performed to obtain the principal curvatures; specifically, the eigenvalue decomposition of the curvature matrix is performed to solve the characteristic equation , obtaining the eigenvalues and the corresponding eigenvectors . Among them, and are the minimum and maximum eigenvalues, respectively, and due to local developability, , the principal curvatures , are given by the eigenvectors and .

[0081] 47) The average curvature of the vertex is determined according to the principal curvatures, and the calculation formula is as follows:​

[0082] .

[0083] average curvature reflects the overall bending degree of the surface at the vertex, the sign of which is determined by the sign of represents a convex region, represents a concave region.

[0084] 48) Based on the average curvature of all vertices in the set of triangular patch vertices, the average curvature of the vertices is extended to the triangular patch by the barycentric coordinate interpolation method to obtain the curvature information of each triangular patch.

[0085] 49) Map all the triangular patch curvature information to RGB color values and assign the two-dimensional unit disc to obtain a curvature map, wherein the convex region and the concave region can be visualized by color difference.

[0086] Step 104, constructing a multi-modal skull gender identification network; the multi-modal skull gender identification network includes a curvature independent main network and a normal-space position shared main network arranged in parallel, a cross-modal bidirectional cross-attention network and a multi-scale fusion classification network connected with the two main networks. Through the cooperative work of the above networks, the comprehensive capture and efficient use of skull morphological features are ensured, and strong feature support is provided for gender identification.

[0087] To fully tap the uniqueness and relevance of different modalities, a double-main feature extraction network is provided in the present application: a curvature independent main network and a normal-space position shared main network. Among them, the curvature independent main network is used to process the curvature map. This design is to introduce the prior knowledge of skull morphology - curvature as the core index to describe the concave-convex changes of the skull surface, which has clear morphological differences in the gender binary significant anatomical region; the normal-space position shared main network is used to share weights to process the normal map and the spatial position map, and to strengthen the inherent correlation between the two in geometric topology description (both reflect the spatial structure attribute of the skull surface).

[0088] The normal-space position shared main network includes a normal subnetwork and a spatial position subnetwork, and the weights of the normal subnetwork and the spatial position subnetwork are shared; as Figure 2 shown, the structure of the curvature independent main network, the normal subnetwork and the spatial position subnetwork is the same, all of which adopt a hierarchical Transformer architecture based on a moving window, and build a pyramid feature level through multi-stage feature extraction.

[0089] Taking the curvature independent main network as an example, as Figure 3As shown, the curvature-independent backbone network includes input preprocessing modules arranged sequentially ( Figure 3 (Not shown in the diagram) The attention modules include a patch partitioning module, a linear embedding module, a first attention module, a second attention module, a third attention module, and a fourth attention module; in addition, a normalization module can be added as needed. Each of the four attention modules includes two sequentially configured moving-window multi-head self-attention sub-modules.

[0090] The input preprocessing module is used to map the geometric and texture information of the curvature map to a regular pixel grid using a uniform sampling algorithm to obtain a curvature pixel map; wherein, the curvature map is a two-dimensional unit disk vector image with color texture, and its numerical representation is: the vertex set is The set of triangular facets is Each piece of dough Indicates the origin of the vertex The resulting triangular topological structure contains its color texture value information. .

[0091] To adapt to network input requirements, the input preprocessing module can be considered a vector image downsampling module, realizing the conversion from vector images to pixel images and size normalization. The core idea of ​​this module is based on the topological relationship between vertices and faces, using a uniform sampling algorithm to map the geometric and texture information of the vector image to a regular pixel grid. The specific steps are as follows: Based on a set of triangle faces... The sampling region is divided, with each facet serving as an independent sampling unit. Sampling points are uniformly generated within each facet. The texture value of each sampling point is calculated using barycentric coordinate interpolation based on the coordinates of the facet's three vertices and the texture value. Finally, the inputs for all three modalities are normalized to a 512×512 pixel size, forming three sets of input tensors: curvature maps. Spatial location map (Record the three-dimensional spatial coordinates of each point on the skull surface) and normal diagram (Describes the direction of the normal vector on the surface of the skull). These standardized inputs lay the foundation for subsequent feature extraction and cross-modal interaction.

[0092] The patch partitioning module is used to divide the curvature pixel map into multiple image patches; specifically, the input image is divided into 4×4 non-overlapping image patches. In particular, for a 512×512 input image, a total of [number missing] patches can be generated. One patch.

[0093] The linear embedding module is used to perform a mapping transformation from pixel space to feature space on the image patch to obtain a sequence feature vector; specifically, linear projection is used to convert the pixel values ​​of each patch into a 128-dimensional feature vector, realizing the mapping from pixel space to feature space. Let the... The pixel value of each patch is Its linear embedding process can be represented as: ;in, This means flattening a 4×4×3 patch into a 48-dimensional vector. The projection weight matrix is... For bias terms, That is, the first The initial feature vectors of each patch. The feature vectors of all patches constitute the initial feature map. The pixel map is transformed into sequence features suitable for Transformer processing through the above patch segmentation and linear embedding.

[0094] like Figure 4 The diagram shows the connection of the first attention module. Each moving window multi-head self-attention submodule includes a first normalization layer, a multilayer perceptron, a second normalization layer, and an attention mechanism layer arranged sequentially. The moving window multi-head self-attention submodule is used to introduce a multi-head self-attention mechanism into the sequence feature vector, and, in conjunction with the window movement strategy, determine the curvature feature block.

[0095] To efficiently capture local spatial correlations in feature maps, this application employs a multi-head self-attention mechanism based on moving windows. This mechanism divides the feature map into 7×7 local windows and calculates self-attention only within these windows, significantly reducing computational complexity. Simultaneously, it enhances information interaction between windows through a window-moving strategy (offsetting the window position by half a window size between adjacent stages). The calculation process is as follows:

[0096] .

[0097] in, These are query, key, and value matrices (obtained by linear transformation of the feature map), used to capture the correlation of features at different locations; The dimension of the key vector (here) This is used to normalize the attention score and prevent numerical overflow; To achieve continuous positional deviations at logarithmic intervals, the network's ability to model spatial structures is enhanced by dynamically adjusting the weights of different relative positions within the window.

[0098] To construct a pyramid-shaped feature hierarchy, each feature extraction stage (except the last stage) includes a patch fusion operation. This means setting up a patch fusion submodule to achieve a halving of resolution and a doubling of the number of channels by merging features from four adjacent patches. In other words, it fuses the curvature feature blocks output by the moving window multi-head self-attention submodule. Let the... The resolution of the stage feature map is , the number of channels is , the first stage after fusion is . This design enables the network to capture features at different scales, from local details at low stages (such as eyebrow arch edges) to global structures at high stages.

[0099] In addition, the normal-space position shared backbone network adopts a 4-stage architecture, sharing weights to process the normal map and the spatial position map . For the normal map, the initial feature is generated by Patch embedding, and then high semantic features are gradually generated through 4 stages of feature extraction: ; where represents the feature processing layer of the stage, which includes 2 mobile window self-attention blocks and 1 Patch fusion sub-module (no Patch fusion for the last stage). The feature extraction process of the spatial position map is similar to the normal Figure 1 : ; since both the normal map and the spatial position map describe the geometric topological properties of the skull surface (the normal vector reflects the surface orientation, and the spatial position reflects the three-dimensional coordinates), weight sharing can strengthen the feature association between the two, improving the overall modeling ability of the skull spatial structure. The feature parameters of each stage are: the first attention module Stage 1 (128x128, 128 channels), the second attention module Stage 2 (64x64, 256 channels), the third attention module Stage 3 (32x32, 512 channels), and the fourth attention module Stage 4 (16x16, 1024 channels).

[0100] The curvature independent backbone network adopts the same 4-stage architecture as the normal-space position shared backbone network, but the parameters are independently initialized and specifically process the curvature map . As a key indicator describing the concave-convex changes of the skull surface (such as the significant difference in male eyebrow arch curvature from female), curvature needs to be learned independently for its unique patterns. Its feature extraction process is: ; where is the initial embedding feature of the curvature map, and the output features of each stage are the same as those of the normal-space position shared backbone network , , maintaining the same resolution and channel number to ensure the feasibility of subsequent cross-modal interaction.

[0101] The core advantage of the dual backbone design set in this application is that through the collaborative strategy of "independent modeling-shared learning", both the unique discriminability of curvature features and the inherent correlation of normal and spatial position features are strengthened, solving the problem of single backbone network that is difficult to balance the differences between multiple modalities.

[0102] To realize the deep interaction and information complementarity of the dual-main features, a cross-modal bidirectional cross-attention guiding mechanism is introduced in each stage (Stage 1 to Stage 4) of feature extraction, so that the curvature independent main network and the normal-space position shared main network can guide each other in the feature learning process. The shared main network provides spatial structure constraints for the curvature features, and the curvature main network supplements the surface morphology details for the shared features, breaking through the static limitations of traditional multi-modal fusion such as "early splicing" or "late fusion". Specifically, the cross-modal bidirectional cross-attention network includes four bidirectional guiding modules.

[0103] The bidirectional guiding module is used to: receive curvature features, normal features and spatial position features corresponding to the feature extraction stage; based on the normal features and the spatial position features, perform feature learning with the curvature features as a reference to obtain a first learning query result; through a residual connection, the first learning query result is fused with the normal features and the spatial position features respectively, and then fed back to the normal sub-network and the spatial position sub-network; based on the curvature features, perform feature learning with the normal features and the spatial position features as a reference to obtain a second learning query result; through a residual connection, the second learning query result is fused with the curvature features, and then fed back to the curvature independent main network.

[0104] Among them, the process of normal features and spatial position features taking curvature features as a reference to absorb surface morphology information described by curvature includes: let the i-th normal feature be The normal feature of the stage shared main is , the spatial position feature is , the feature of the curvature main is , and the query process is:

[0105] .

[0106] .

[0107] Among them, is a cross-attention function, which is similar in calculation to self-attention, but the keys (K) and values (V) come from heterogeneous modal features, specifically:

[0108] .

[0109] Through this process, the normal features can integrate the surface concave-convex patterns described by the curvature (such as the influence of the cheekbone curvature on the direction of the normal vector), and the spatial position features can associate the spatial distribution of the curvature changes (such as the correspondence between the location of the jaw angle and the curvature mutation).

[0110] The curvature feature takes the normal feature and the spatial position feature as reference, absorbs the geometric topological information to optimize the process of its own representation:

[0111] .

[0112] .

[0113] Through this process, the curvature feature can combine the direction information of the normal vector (such as the convergence mode of the normal vector corresponding to the area with positive curvature) with the spatial coordinate constraint (such as the range of curvature change of the glabella bone region is limited by its spatial position), to improve the discrimination ability of key anatomical parts.

[0114] To ensure that the original feature information is not covered, the cross-attention output is fused with the original feature through a residual connection to realize dynamic updating of the feature: ; ; ; This updating mechanism enables the feature to dynamically absorb cross-modal complementary information while retaining its core information.

[0115] In summary, the progressive optimization of cross-modal bidirectional cross-attention guidance in the bidirectional guidance module (from Stage 1 to Stage 4) enables the feature interaction to gradually upgrade from low-level local detail association to high-level global semantic collaboration, ultimately forming a comprehensive feature representation with multi-modal information.

[0116] The multi-scale fusion classification network is used for: feature fusion and classification output of the final curvature feature output by the curvature independent backbone network, the final normal feature output by the normal sub-network and the final spatial position feature output by the spatial position sub-network. Specifically, after processing by the above-mentioned double backbone network, three groups of high-level features can be obtained: normal feature , spatial position feature (resolution 16x16, channel 1024) and curvature backbone (same specification). To make full use of these features for gender determination, the present application designs a multi-scale feature fusion classification network, which realizes the optimal integration of features through high-level feature refinement, global integration and channel weight learning, and improves the discrimination ability of the model. Specifically, the processing steps realized in the multi-scale fusion classification network corresponding to the integrated decision fusion module described in Figure 2 are as follows:

[0117] Fine fusion of normal feature and spatial position feature: taking normal feature as query (Q) and value (V), and spatial position feature as key (K), the cross-attention mechanism is used to strengthen the weight of the normal feature in the key anatomical part: ; where the residual connection ensures the original discriminability of normal features is not lost, and the cross-attention guides the focus of normal vectors to the regions with significant gender differences by using the coordinate information of spatial position features, so that the fused features Simultaneously possess directional information and spatial positioning ability.

[0118] Then, the above fine-grained fused features and the high-level features of the curvature backbone are integrated to form global feature representation: This integration method is based on the complementarity of feature spaces: Focusing on the spatial structure and normal distribution of the skull, Highlighting the surface curvature change pattern, the superposition of the two realizes the complete description of "morphological structure + surface characteristics", providing comprehensive feature support for gender identification. That is, after feature integration, the final features obtained by integration are sent to the classifier for classification, so as to obtain the gender and estimated gender probability of three-dimensional skull determination.

[0119] To further improve the discriminability of features, a channel attention mechanism (SE-Net) is introduced to dynamically learn the importance weights of each channel, and to strengthen the feature channels that make significant contributions to gender identification. It includes global average pooling, channel weight learning, weighted feature generation, and classification output.

[0120] Global average pooling is: global average pooling is performed on the global features (size ) to compress the spatial dimension information into channel dimension statistics:

[0121] .

[0122] Where, is the global average response of the th channel, reflecting the average contribution of the channel features in the entire space.

[0123] Channel weight learning is: through two fully connected layers, the dependency relationship between channels is learned to generate normalized channel weights: ; where, and are the fully connected layer weights ( is the compression ratio), and are used to introduce nonlinearity and weight normalization, respectively, is the importance weight of the th channel.

[0124] Weighted feature generation is: multiply the learned channel weights with the global features channel by channel to strengthen the feature contribution of key channels: .

[0125] The classification output is: after LayerNorm and adaptive average pooling of the weighted features, the linear classification head generates a gender prediction result: .

[0126] In addition, to improve the robustness of the model, the application additionally designs an auxiliary classification branch to separately classify the high-level features of the curvature backbone and strengthen the learning quality of the curvature features through an auxiliary loss: Based on the auxiliary classification branch, the total loss in the training process is the weighted sum of the cross-entropy main loss and the auxiliary loss: Wherein, is the main loss (Y is the true label), is the auxiliary loss, and is the balance coefficient. The auxiliary loss forces the curvature backbone to learn more discriminative features through deep supervision, while regularizing the main branch to reduce the risk of overfitting.

[0127] To achieve high-precision skull gender identification, the application designs a multi-modal skull gender identification network based on morphological feature guidance. The network aims to fuse the curvature map, spatial position map (XYZ) and normal Figure 3 vector map of the skull, deeply mine the morphological features of the skull, effectively fuse the multi-modal information, and provide strong technical support for high-precision skull gender identification, breaking through the limitations of traditional single-modal feature learning.

[0128] Step 105, input the normal map, spatial position map, curvature map and gender label corresponding to each three-dimensional skull model in the three-dimensional skull data set into the multi-modal skull gender identification network for training to obtain a skull gender identification model; wherein the skull gender identification model is used to output the corresponding gender when receiving the normal map, spatial position map and curvature map of the target three-dimensional skull model as input.

[0129] During training, the main parameter configuration of the network is as follows: the input image size is 512x512x3, the patch size is 4x4, and the initial embedding dimension is 128; the number of channels in the four feature extraction stages is 128, 256, 512, and 1024 respectively, and each stage contains 2 mobile window self-attention blocks (Stage 3 is 16 to enhance feature learning ability); the number of cross-attention heads matches the number of channels in the corresponding stage (3, 6, 12, 24), and the channel attention compression ratio is 8, .

[0130] ​In the implementation process, the network is built using the PyTorch framework, the AdamW optimizer (initial learning rate , weight decay ) is used, the training batch size is 4, the pre-trained weights based on the single-main version trained on the same single-small-scale dataset are loaded to expect the model to converge quickly, the total iteration number is 200 epochs, and the cosine annealing strategy is used to adjust the learning rate. Experiments show that the network can be efficiently trained on a single NVIDIA RTX4070 graphics card, with an inference time of about 83ms / sample, balancing the accuracy and efficiency requirements.

[0131] To fully verify the advancement of the method, it is compared with the mainstream skull gender identification methods and traditional machine learning algorithms. The comparison experiment uses the same performance evaluation indicators (accuracy, AUC value) to ensure the comparability and objectivity of the results.

[0132] In comparison with existing skull gender identification research, the method shows significant advantages. As shown in Table 1, on a dataset containing 420 samples, the gender identification accuracy of the method reaches 97.6%, and the AUC value is 0.98, which is better than most of the comparison methods. Among them, the improved back propagation neural network achieves an accuracy of 96.8% on 267 samples, but relies on manual feature extraction; the deep feature selection and fusion strategy achieves an accuracy of 96.4% on 421 samples, while the method further improves the performance through conformal mapping and multi-modal fusion. It is worth noting that although some studies have achieved an accuracy of 94.4% on a small sample dataset (90 samples), their generalization ability is questionable due to the limited sample size. The method achieves high accuracy and high AUC value on a medium-sized dataset, proving its advantages in balancing sample efficiency and identification performance.

[0133] Table 1

[0134]

[0135] The comparison experiment with traditional machine learning methods further verifies the superiority of the deep learning architecture. As shown in Table 2, on the same dataset, the method significantly outperforms SVM (72.9% / 0.828), KNN (63.0% / 0.630), random forest (78.5% / 0.722), and other traditional algorithms with an accuracy of 97.6% and an AUC value of 0.984. Among them, the KNN algorithm performs the worst, possibly because it cannot capture the complex nonlinear relationship of skull features; although SVM can handle nonlinear problems through kernel functions, it still performs worse than deep learning methods in high-dimensional feature space. Traditional machine learning methods rely on manual feature engineering, while the method effectively mines the potential discriminant patterns of skull morphology through automated feature extraction and multi-modal fusion.

[0136] Table 2

[0137]

[0138] The present application proposes a solution based on conformal mapping and multi-modal fusion learning to the technical difficulties of skull gender identification. The effectiveness and innovation of the method are verified through systematic experiments. The large-scale three-dimensional skull dataset constructed in the present application covers multiple regions and multiple nationalities, providing a reliable data basis for research. The automatic conformal mapping method successfully converts the three-dimensional skull surface into a two-dimensional vector map that retains topological and geometric features, solving the problem of high-dimensional data processing. The generation of normal maps, curvature maps, and space XYZ maps effectively retains gender binary characteristics. The dual-main architecture of the bidirectional guided multi-modal learning network realizes the synergy of independent modeling of curvature features and shared learning of normal-space features. The cross-modal bidirectional cross-attention mechanism promotes dynamic interaction of features, and the multi-scale fusion module enhances the discriminability of key features.

[0139] The experimental results show that the method has an accuracy of 97.6% and an average AUC of 0.98, and performs stably in cross-national and cross-regional samples. The gradient heat map shows that the present application accurately captures the differences in key areas such as the eye socket and the brow. The present application not only improves the accuracy and generalization ability of skull gender identification, but also provides an important reference for the field of three-dimensional biological feature recognition.

[0140] In summary, the present application has the following advantages compared to the prior art:

[0141] (1) A large-scale three-dimensional skull and two-dimensional vector mapping dataset is constructed: To effectively address the small sample size problem of existing identification methods, head CT scan data from individuals of different ages and genders in multiple regions is collected and standardized registration processing is performed on these scan results. A high-quality, widely representative three-dimensional skull dataset is finally constructed. On this basis, an additional two-dimensional vector mapping dataset of the cranial anterior surface with significant morphological features is constructed. This provides rich morphological information and a reliable analysis basis for subsequent research.

[0142] (2) An automatic processing method for two-dimensional feature images of three-dimensional cranial surfaces is constructed: To address the difficulty of high-dimensional feature extraction and analysis, the present method uses conformal mapping technology to conformally map the three-dimensional skull surface to a two-dimensional unit disc. Based on the vertex coordinates, normal vectors, and curvature information of the three-dimensional model, two-dimensional feature images of the cranial surface are constructed, effectively strengthening the key morphological features of the cranial surface and reducing the data dimension. This simplifies the feature extraction process, reduces the computational load, and makes gender-related features more easily trained by deep learning networks, thereby providing an efficient and representative feature vector for subsequent models.

[0143] (3) Set up a double-main feature extraction architecture: In view of the characteristics of multi-modal features, a curvature independent main trunk and a normal-space position shared main trunk are designed, the feature uniqueness and relevance are balanced through the "independent modeling-shared learning" strategy, and the problem that a single main network is difficult to balance multi-modal differences is solved. A cross-modal bidirectional cross-attention guide is set up: a bidirectional interaction mechanism is introduced at each feature extraction stage, so that the double-main features can dynamically absorb cross-modal complementary information, progressive information fusion from local details to global structure is realized, and the static limitation of traditional fusion methods is broken through. Multi-scale adaptive fusion and deep supervision are set up: through high-level feature refinement, channel attention weighting and auxiliary loss design, the optimal integration and key feature enhancement of multi-modal features are realized, and the capture ability and generalization performance of the model to gender difference features are improved. These designs jointly build an end-to-end learning framework that can deeply mine the features of skull morphology and effectively fuse multi-modal information, providing strong technical support for high-precision skull gender identification.

[0144] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram thereof can be as shown in Figure 5 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement the multi-modal based skull gender identification method.

[0145] Those skilled in the art can understand that Figure 5 the structure shown in the above

[0146] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in each of the above method embodiments.

[0147] In an exemplary embodiment, a computer readable storage medium storing a computer program is provided, the computer program, when executed by a processor, implements the steps of any of the above method embodiments.

[0148] In an exemplary embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of any of the above method embodiments.

[0149] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0150] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0151] The database involved in each of the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each of the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0152] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as there is no contradiction.

[0153] The principles and implementation manners of the present application are described by applying specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for skull gender identification based on multi-modal, characterized in that, The method comprises: reconstructing a three-dimensional skull surface model based on skull CT scan data to obtain a three-dimensional skull data set; for each three-dimensional skull surface model in the three-dimensional skull data set, using Koebe iteration to realize conformal mapping from a three-dimensional multi-connected region to a plane circular region to obtain a two-dimensional unit disc; respectively performing normal calculation and coloring, vertex coordinate calculation and coloring, and curvature calculation and coloring on the two-dimensional unit disc to obtain a normal map, a spatial position map, and a curvature map; constructing a multi-modal skull gender identification network; the multi-modal skull gender identification network comprises a curvature independent main network and a normal-spatial position shared main network arranged in parallel, a cross-modal bidirectional cross-attention network and a multi-scale fusion classification network connected with the two main networks; the normal-spatial position shared main network comprises a normal subnetwork and a spatial position subnetwork, and the normal subnetwork and the spatial position subnetwork share weights; the curvature independent main network, the normal subnetwork, and the spatial position subnetwork have the same structure; the curvature independent main network comprises an input preprocessing module, a Patch division module, a first attention module, a second attention module, a third attention module, and a fourth attention module arranged in sequence; wherein the four attention modules each comprise a linear embedding submodule and two moving window multi-head self-attention submodules arranged in sequence; the input preprocessing module is configured to map the geometric and texture information of the curvature map to a regular pixel grid by using a uniform sampling algorithm to obtain a curvature pixel map; the Patch division module is configured to divide the curvature pixel map into a plurality of image blocks; the linear embedding submodule is configured to perform mapping conversion from a pixel space to a feature space on the image blocks to obtain a sequence feature vector; and the moving window multi-head self-attention submodule is configured to introduce a multi-head self-attention mechanism for the sequence feature vector, and determine a curvature feature block in combination with a window moving strategy; inputting the normal map, the spatial position map, the curvature map, and the gender label corresponding to each three-dimensional skull surface model in the three-dimensional skull data set into the multi-modal skull gender identification network for training to obtain a skull gender identification model; wherein the skull gender identification model is configured to output a corresponding gender when receiving the normal map, the spatial position map, and the curvature map of a target three-dimensional skull surface model as input.

2. The multi-modal based skull gender identification method of claim 1, wherein, Reconstructing a three-dimensional skull surface model based on skull CT scan data to obtain a three-dimensional skull data set comprises: for each set of skull CT scan data, performing smoothing processing using a Gaussian filter, and extracting the outline of the skull and skin using a Sobel edge detection operator; based on the outline of the skull and skin, performing three-dimensional reconstruction of the skull and skin using a Marching Cubes algorithm to obtain a three-dimensional reconstruction model; performing smoothing processing on the three-dimensional reconstruction model to obtain an initial three-dimensional skull model; adjusting all initial three-dimensional skull models corresponding to the skull CT scan data to a preset coordinate system, and then cutting to obtain a plurality of skull frontal view surfaces; Align and normalize the anterior surface of the skull as a three-dimensional skull surface model to obtain a three-dimensional skull data set.

3. The multi-modal based skull gender identification method of claim 1, wherein, For each three-dimensional skull surface model in the three-dimensional skull data set, a Koebe iteration is used to realize conformal mapping from a three-dimensional multi-connected region to a planar circular region to obtain a two-dimensional unit disc, including: Boundary extraction is performed on the three-dimensional skull surface model to obtain bilateral orbital rim boundaries, piriform aperture boundaries, bilateral temporal sphenoid-zygomatic arch gap boundaries, bilateral mandibular-zygomatic arch gap boundaries, and curved surface outer contour boundaries; Seven holes are determined based on the bilateral orbital rim boundaries, the piriform aperture boundaries, the bilateral temporal sphenoid-zygomatic arch gap boundaries, and the bilateral mandibular-zygomatic arch gap boundaries; Any one of the seven holes is selected as a current hole, and all holes except the current hole are filled to obtain a topological ring surface; A dynamic Yamabe flow is used to calculate the conformal mapping of the topological ring surface to a standard planar circular region to map the current hole to a circular hole; Until the mapping of the seven holes is completed, a two-dimensional unit disc with multiple circular holes conformally equivalent to the three-dimensional skull surface model is obtained.

4. The multi-modal based skull gender identification method of claim 3, wherein, The function formula of the dynamic Yamabe flow is: ； wherein, is a discrete conformal factor, for determining the degree of deformation of the curved surface in the conformal transformation; V is a set of triangle mesh vertices of a three-dimensional skull curved surface model, is a set of real numbers; is the target curvature of the i-th triangle mesh vertex, which refers to the vertex curvature value expected to be achieved; is the current discrete Gaussian curvature of the i-th triangle mesh vertex at time .

5. The multi-modal based skull gender identification method of claim 3, wherein, Filling all holes except the current hole to obtain a topological ring surface includes: Selecting any hole except the current hole as a to-be-filled hole; The center point of the to-be-filled hole is determined by a centroid calculation-based method; The center point and each vertex of the to-be-filled hole are sequentially connected to obtain a set of line segments; New triangular patches are determined based on the set of line segments and the boundary of the to-be-filled hole, combined with the constraint of the triangle inequality; The to-be-filled hole is filled based on the new triangular patches; Until all holes except the current hole are filled to obtain a topological ring surface. 6.The multi-modal based skull gender identification method of claim 1, wherein, Normal calculation and coloring, vertex coordinate calculation and coloring are performed on the two-dimensional unit disc to obtain a normal map and a spatial position map, including: Based on the two-dimensional unit disc, a triangular patch set of the corresponding three-dimensional skull surface model is determined; For each triangular patch in the triangular patch set, two edge vectors are calculated based on any vertex, and then the two edge vectors are cross-multiplied to obtain a normal vector; For each triangular patch in the triangular patch set, the patch barycenter coordinates are calculated by arithmetic mean of the coordinates of the three vertices; The normal vector and the patch barycenter coordinates are normalized respectively; The normalized normal vector is mapped to an RGB color value by interval conversion, and is assigned to the two-dimensional unit disc to obtain a normal map; The normalized patch barycenter coordinates are mapped to an RGB color value by interval conversion, and are assigned to the two-dimensional unit disc to obtain a spatial position map.

7. The multi-modal based skull gender identification method of claim 1, wherein, Curvature calculation and coloring are performed on the two-dimensional unit disc to obtain a curvature map, including: Based on the two-dimensional unit disc, a triangular patch vertex set of the corresponding three-dimensional skull surface model is determined; For each vertex in the triangular patch vertex set, a geodesic neighborhood is determined; calculating a neighborhood area of the geodesic neighborhood based on the vertices within the geodesic neighborhood; calculating a geometric property of each edge based on the edges within the geodesic neighborhood; calculating a curvature matrix based on the neighborhood area of the geodesic neighborhood and the geometric property of each edge; performing eigenvalue decomposition on the curvature matrix to obtain principal curvatures; determining average curvatures of the vertices according to the principal curvatures; extending the average curvatures of the vertices to the triangular patches by barycentric coordinate interpolation method based on the average curvatures of all the vertices in the triangular patch vertex set to obtain triangular patch curvature information; mapping all the triangular patch curvature information to RGB color values and assigning the two-dimensional unit disc to obtain a curvature map.

8. The multi-modal based skull gender identification method of claim 7, wherein, the curvature matrix of the vertex i is: ； wherein, is the area of the neighborhood, B is the geodesic neighborhood, denotes the edge measured geodesic neighborhood the effective length of the intercept; denotes the edge the angle between the normals of the two triangular facets, ranging from the sign determines the positive or negative; is the outer product matrix of the unit vectors with elements ; the unit vectors are calculated from the edge e as follows: ； ； Among them, the edge From the endpoint and Sure, For the edge The Euclidean length. 9.The multi-modal based skull gender identification method of claim 1, wherein, The cross-modal bidirectional cross-attention network comprises four bidirectional guide modules; The bidirectional guide module is used to correspond to the received curvature feature, normal feature and spatial position feature in the feature extraction stage; based on the normal feature and the spatial position feature, feature learning is performed with the curvature feature as a reference to obtain a first learning query result; through residual connection, the first learning query result is fused with the normal feature and the spatial position feature respectively, and then fed back to the normal subnetwork and the spatial position subnetwork; based on the curvature feature, feature learning is performed with the normal feature and the spatial position feature as a reference to obtain a second learning query result; through residual connection, the second learning query result is fused with the curvature feature, and then fed back to the curvature independent backbone network; The multi-scale fusion classification network is used for: feature fusion and classification output of the final curvature feature output by the curvature independent backbone network, the final normal feature output by the normal subnetwork and the final spatial position feature output by the spatial position subnetwork. The cross-modal bidirectional cross-attention network comprises four bidirectional guide modules; The bidirectional guide module is used to correspond to the received curvature feature, normal feature and spatial position feature in the feature extraction stage; based on the normal feature and the spatial position feature, feature learning is performed with the curvature feature as a reference to obtain a first learning query result; through residual connection, the first learning query result is fused with the normal feature and the spatial position feature respectively, and then fed back to the normal subnetwork and the spatial position subnetwork; based on the curvature feature, feature learning is performed with the normal feature and the spatial position feature as a reference to obtain a second learning query result; through residual connection, the second learning query result is fused with the curvature feature, and then fed back to the curvature independent backbone network; The multi-scale fusion classification network is used for: feature fusion and classification output of the final curvature feature output by the curvature independent backbone network, the final normal feature output by the normal subnetwork and the final spatial position feature output by the spatial position subnetwork.

Images