Double-layer hypergraph feature fusion sketch part three-dimensional model cross-domain retrieval method

Abstract

The invention provides a sketch part three-dimensional model cross-domain retrieval method based on double-layer hypergraph feature fusion, and belongs to the technical field of part three-dimensional model retrieval, and the method comprises the following steps: S1, projection sketch generation and selection; s2, part three-dimensional model feature extraction; s3, cross-domain feature matching is carried out; and S4, performing end-to-end training and retrieval. According to the method, through complexity and difference selection of the projection sketches, it is ensured that key information of parts is completely reserved and is free of redundancy; the deep residual network fusing the attention mechanism synchronously extracts global and local key features, and the double-layer hypergraph neural network realizes multi-view feature deep fusion and semantic enhancement; the multi-loss metric learning eliminates cross-domain difference, and the end-to-end training framework realizes collaborative optimization of each module, so that the retrieval accuracy and recall rate are remarkably improved, and the demand of efficiently reusing a part model in the manufacturing industry is met.

Description

Technical Field

[0001] This invention relates to the field of 3D model retrieval technology for parts, and in particular to a cross-domain retrieval method for sketched 3D models of parts using a two-layer hypergraph feature fusion approach. Background Technology

[0002] With the widespread adoption of Computer-Aided Design (CAD) systems in the manufacturing industry, companies have accumulated a massive number of 3D models of parts. Designers need to reuse existing models in new product development to shorten the cycle, but traditional text or geometric feature retrieval suffers from problems such as descriptive ambiguity and inaccurate feature capture. As hand-drawn sketches are a commonly used tool for designers to express their ideas, sketch-based 3D model retrieval of parts has become a core requirement.

[0003] In recent years, research on sketch-based 3D model retrieval methods has increased. Early methods extracted features from sketches and 3D models manually. Due to limited feature discrimination capabilities, these methods could not objectively and comprehensively extract common features from sketches and 3D models, resulting in poor alignment between the two modalities and low retrieval accuracy.

[0004] In recent years, deep learning models have been successfully applied in many fields, driving the development of sketch-based 3D model retrieval. Generally speaking, these methods mainly focus on three aspects: key view selection, feature representation of 3D models, and feature similarity matching.

[0005] However, current methods for key view selection, 3D model feature representation, and feature similarity matching all have shortcomings, as detailed below: 1. Existing view selection methods are mainly aimed at ordinary 3D models in daily life, without fully considering the unique properties of complex structures such as microholes, chamfers, and threads in the manufacturing field. As a result, the selected views are difficult to balance in terms of information content, overall shape and local details.

[0006] 2. Existing feature fusion methods for 3D part models mainly use strategies such as simple stitching and weighted fusion to overlay features from multiple views. These methods ignore the complementarity and differences between different views, and cannot collaboratively express and learn the complex multi-dimensional relationships between "view-part-part class", thus limiting the overall expressive power of the 3D part model.

[0007] 3. Existing cross-domain matching methods cannot balance global consistency and fine-grained discriminativeness with a single loss function, resulting in a decrease in discriminative power. They are difficult to effectively align cross-modal features between the 3D model of the part and the single-view sketch, and the training mechanism lacks dynamic adaptability to collaboratively optimize the learning process from global initialization to local refinement.

[0008] Therefore, there is an urgent need to design a sketch-based method for retrieving 3D models of parts to improve retrieval efficiency and quality. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings and deficiencies of the existing technology and to provide a cross-domain retrieval method for sketched part 3D models by fusing two-layer hypergraph features.

[0010] The technical solution adopted by this invention is as follows: This application provides a cross-domain retrieval method for sketched 3D models of parts using a two-layer hypergraph feature fusion, comprising the following steps: S1: Projection sketch generation and selection, obtaining multiple projection views by multi-angle projection of the 3D model of the part, converting each projection view into a corresponding projection sketch, performing complexity analysis and difference calculation on the projection sketches, and iteratively selecting to obtain a set of projection sketches representing the 3D model of the part; S2: Feature extraction of the 3D model of the part, using a deep residual network with a fusion attention mechanism to extract the global features and local key features of each projection sketch in the set of projection sketches, and then... Feature extraction and fusion are performed using a two-layer hypergraph neural network to obtain deeply fused and semantically enhanced 3D model features of the part; S3: Cross-domain feature matching, a metric learning model based on multiple losses is constructed to map the features of the hand-drawn sketch and the features of the 3D model of the part to the same metric space to achieve cross-domain feature alignment, and cross-domain matching is completed by calculating similarity based on the aligned features; S4: End-to-end training and retrieval, an end-to-end training framework with a joint loss function is built to connect the projection sketch generation and selection, the feature extraction of the 3D model of the part and the cross-domain feature matching operation. The joint loss function dynamically and collaboratively optimizes the operation parameters and outputs the retrieval results of the target 3D model of the part.

[0011] In some embodiments, in step S1, each of the projected views is converted into a corresponding projected sketch, specifically including: converting each of the projected views into a corresponding projected sketch by incorporating an edge detection algorithm; the edge detection algorithm includes the Sobel edge detection algorithm and the Canny edge detection algorithm; the sketching process is: extracting the contour edges of the projected view by the edge detection algorithm, performing line transformation on the contour edges, and generating the projected sketch that conforms to the hand-drawn sketch modality.

[0012] In some embodiments, step S1 involves performing complexity analysis and difference calculation on the projected sketches, specifically including: complexity analysis: constructing a complexity evaluation system that includes information entropy, edge density, and contour features, and calculating the complexity value of each projected sketch; selecting the projected sketch with the highest overall complexity as an initial candidate sketch; difference calculation: constructing feature vectors for the initial candidate sketch and other projected sketches to be screened based on the information entropy, edge density, and contour features, respectively, and calculating the cosine similarity between the two types of feature vectors to evaluate the difference between the projected sketches; introducing the information entropy of the projected sketches and normalizing it, using the normalized information entropy as an information content index, correcting the difference evaluation results, and iteratively selecting until a set of projected sketches of a preset size is obtained.

[0013] In some embodiments, the process of obtaining the contour features is as follows: first, nonlocal mean denoising is performed on the projected sketch to eliminate noise interference, and then the main contour of the denoised projected sketch is extracted; the geometric parameters of the main contour are calculated, including roundness, aspect ratio and convexity, and the geometric parameters are normalized and nonlinearly processed respectively to obtain the contour features used for complexity analysis.

[0014] In some embodiments, in step S2, the deep residual network with fused attention mechanism includes: using a ResNet series deep residual network as the base network, and integrating a convolutional block attention module in the feature extraction link of the base network; the convolutional block attention module includes a channel attention submodule and a spatial attention submodule, which are cascaded; the channel attention submodule enhances the feature channel response corresponding to the key structure of the part's 3D model by analyzing the importance of each feature channel; the key structure is a special geometric feature on the 3D model; the spatial attention submodule captures local detail information by locating the spatial position of local key features in the projection sketch, thereby achieving synchronous extraction of global features and local key features.

[0015] In some embodiments, in step S2, the feature extraction and fusion of the two-layer hypergraph neural network includes: view-level hypergraph processing: using the combined features of global features and local key features of each projection sketch corresponding to the same part 3D model as nodes, constructing hyperedges for each part 3D model according to the part affiliation; dynamically calculating the node weight of each node based on the distance from each node feature to the feature center of the corresponding part 3D model; aggregating all node features through hypergraph convolution operation to obtain the preliminary features of the part 3D model; category-level hypergraph processing: using the preliminary features of multiple part 3D models as nodes, constructing hyperedges for each part category according to the part category label; dynamically calculating the node weight of each node based on the distance from each node feature to the category feature center of the corresponding part category; sequentially performing feature aggregation and feature update operations through hypergraph convolution to output the part 3D model features with high intra-class feature compactness and high inter-class feature discriminability.

[0016] In some embodiments, the hypergraph convolution operation includes: node weighting: weighting the features of corresponding nodes according to the node weights to obtain weighted node features; hyperedge information transfer: transferring all weighted node features connected by the same hyperedge to the hyperedge to realize node feature information interaction within the hyperedge range; feature normalization: introducing the node degree matrix and hyperedge degree matrix of the hypergraph, calculating the inverse matrix of the node degree matrix and hyperedge degree matrix, and normalizing the features after information transfer through the inverse matrix to ensure the fairness and numerical stability of information transfer within the hypergraph.

[0017] In some embodiments, in step S3, the metric learning model of multiple losses includes: Two-branch feature projection: Constructing a two-branch adaptive projection network, where one branch processes the features of the hand-drawn sketch and the other branch processes the features of the 3D model of the part; The two types of features are mapped using the two-branch adaptive projection network, and normalization is performed on the mapped features to obtain standardized feature vectors in the same metric space; Multiple loss construction: Fusing contrastive loss, triplet loss, and cross-domain center loss to form the multiple loss; Based on the multiple loss, a progressive training mechanism is constructed to optimize the metric space, making semantically similar features closer together and semantically different features farther apart; Hybrid matching: A strategy combining category-level matching and instance-level matching is adopted in the retrieval stage; Category-level matching achieves coarse cross-domain matching by evaluating the deviation of the standardized feature vector from the category feature center of the corresponding part category; Instance-level matching achieves fine cross-domain matching by calculating the cosine similarity between the standardized feature vectors.

[0018] In some embodiments, the progressive training mechanism includes three sequentially connected stages: a global initialization stage, where the contrastive loss is used as the primary method to construct the basic feature distribution of the metric space, enabling the initial separation of features of different categories within the metric space; a local optimization stage, where the triplet loss is used as the primary method to constrain the features of different instances of the same type, thereby improving the instance-level feature discrimination capability; the features include features of 3D models of different parts of the same type, features of hand-drawn sketches and 3D models of non-target parts of the same type; and a fine-tuning stage, where online hard sample mining technology is introduced to select easily confused hard samples in the metric space for training, optimize the feature decision boundary, and further improve cross-domain matching accuracy.

[0019] In some embodiments, in step S4, the joint loss function is an integration of classification loss and metric loss; the dynamic collaborative optimization of the parameters of each module includes: dividing the training cycle into early training, middle training and late training based on the training cycle ratio; in the early training, the classification loss is dominant to build basic feature representation capabilities; in the middle training, the weights of the classification loss and metric loss are balanced to achieve collaborative optimization of feature discriminability and cross-domain alignment; in the late training, the metric loss is dominant to focus on fine adjustment of cross-domain feature space; through the gradient backpropagation mechanism, the loss signal generated by the joint loss function is synchronously transmitted to the projection sketch generation and selection, part 3D model feature extraction and cross-domain feature matching modules to achieve collaborative update of all module parameters.

[0020] The beneficial effects of this invention are as follows: By selecting based on the complexity and differences of the projected sketches, this invention ensures that the key information of the parts is completely preserved without redundancy; a deep residual network with an attention mechanism simultaneously extracts global and local key features, and a two-layer hypergraph neural network achieves deep fusion of multi-view features and semantic enhancement; multiple loss metric learning eliminates cross-domain differences, and an end-to-end training framework enables collaborative optimization of each module, significantly improving retrieval accuracy and recall, and meeting the needs of the manufacturing industry for efficient reuse of parts models. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0022] Figure 1 This is a schematic diagram of the method framework of the present invention; Figure 2 This is a schematic diagram of the framework of the method for screening 3D model projection sketches of parts that integrates complexity and variability according to the present invention. Figure 3 This is a schematic diagram of the residual network structure of the fusion attention mechanism for sketch feature extraction according to the present invention; Figure 4 This is a structural diagram of the two-layer hypergraph neural network for feature extraction of 3D models of parts according to the present invention; Figure 5 This is a schematic diagram of the multi-loss metric learning network structure of the present invention; Figure 6 This is a schematic diagram of the end-to-end training framework for retrieving 3D models of parts according to the present invention; Figure 7 This is a schematic diagram comparing the PR curves of different methods of the present invention; Figure 8 This is a schematic diagram of the scatter points for visualizing the features of the part model in this invention; Figure 9 This is a schematic diagram comparing the evaluation index results of the present invention with those of seven typical methods; Figure 10 This is a schematic diagram illustrating the retrieval performance of different numbers of projection sketches according to the present invention; Figure 11 This is a schematic diagram illustrating the retrieval performance of different network structures according to the present invention; Figure 12 This is a schematic diagram illustrating the retrieval performance of the block-based training network and the end-to-end training network of the present invention; Figure 13 This is a schematic diagram illustrating an example of the search results of the present invention; Figure 14 This is a schematic diagram of the retrieval method of the present invention. Detailed Implementation

[0023] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0024] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "longitudinal", "lateral", "radial", "length", "width", "thickness", "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are mainly for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element or component to have a specific orientation, or to be constructed and operated in a specific orientation.

[0025] It should be noted that the terms "first," "second," and similar words do not indicate any order, quantity, or importance, but are only used to distinguish different components and should not be construed as limiting the embodiments of this application.

[0026] It should be noted that the terms "installation," "setup," "equipped with," "connection," and "connected" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral structures; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium, or internal connections between two devices, components, or parts.

[0027] It should be noted that the terms "in some embodiments," "exemplarily," and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "in some embodiments," "exemplarily," or "for example" should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "in some embodiments," "exemplarily," and "for example" is intended to present related concepts in a specific manner, meaning that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the above terms in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. Those skilled in the art will explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0028] Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0029] Regarding the accompanying drawings of this application, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not necessarily drawn to scale.

[0030] Existing sketch-based methods for retrieving 3D part models generally suffer from technical problems such as loss of key information when converting 3D part models to 2D views, insufficient fusion of multi-view features, low cross-domain matching accuracy between hand-drawn sketches and 3D models, and feature space discontinuity caused by modular training, which seriously affect retrieval efficiency and accuracy.

[0031] Based on the above problems, this application proposes a cross-domain retrieval method for sketched part 3D models using two-layer hypergraph feature fusion, such as... Figure 1 As shown, it includes four core modules: a projection sketch representation and filtering method for the 3D model of a part, a feature extraction model for the 3D model of a part based on two-layer hypergraph feature fusion, a cross-domain matching method based on multiple loss metric learning, and an end-to-end training framework for the 3D model of a part based on a joint loss function.

[0032] During the training phase of the retrieval model, the execution steps between the modules are as follows: S1: Projection sketch generation and selection: Multiple projection views are obtained by multi-angle projection of the 3D model of the part. Each projection view is converted into a corresponding projection sketch. Complexity analysis and difference calculation are performed on the projection sketches. Iterative selection is carried out to obtain a set of projection sketches representing the 3D model of the part. S2: Feature extraction of the 3D model of the part. A deep residual network with a fusion attention mechanism is used to extract the global features and local key features of each projection sketch in the projection sketch set. Then, a two-layer hypergraph neural network is used for feature extraction and fusion to obtain the 3D model features of the part with deep fusion and semantic enhancement. S3: Cross-domain feature matching. Construct a metric learning model based on multiple losses to map the features of hand-drawn sketches and the features of the 3D model of the part to the same metric space to achieve cross-domain feature alignment. Calculate similarity based on the aligned features to complete cross-domain matching. S4: End-to-end training and retrieval. An end-to-end training framework with a joint loss function is built to connect the generation and selection of projection sketches, feature extraction of 3D models of parts, and cross-domain feature matching operations. The joint loss function dynamically and collaboratively optimizes the parameters of each operation and outputs the retrieval results of the 3D model of the target part.

[0033] During the operation phase of the retrieval model, the user first inputs a retrieval sketch, and the feature extraction network obtains the features of the retrieval sketch. Then, the retrieval model calculates the similarity between the features of the sketch and the 3D model of the part in a unified depth metric space obtained during training, realizing cross-domain matching between the sketch and the 3D model of the part, and finally returns the retrieval results.

[0034] In some embodiments, regarding step S1: In order to fully express the complex three-dimensional model of the part and support high-precision cross-domain retrieval, this application uses the 12-viewpoint projection method to project the three-dimensional model from multiple angles to obtain the projected view, and uses the mechanical sketching method incorporating Sobel edge detection to convert the projected view into a line-based sketch representation, called the projected sketch.

[0035] Selecting the most representative set of projected sketches to represent the 3D model is crucial for improving retrieval efficiency and accuracy. Considering the complex characteristics of mechanical part models, such as edge complexity, contour features, shape geometry, and texture information, this application proposes a method for selecting projected sketches of 3D part models that integrates complexity and diversity. Figure 2 As shown.

[0036] The complexity of projected sketches is comprehensively measured by combining information entropy, edge density, and contour analysis. First, the information entropy of each projected sketch is calculated using the grayscale histogram of the image. As in equation (1), where, The first element in the normalized grayscale histogram represents the... The probability of a grayscale value. Information entropy measures the amount of information in a projection sketch by analyzing the pixel grayscale distribution. A projection sketch with high information entropy contains more irregular geometric information, such as holes, grooves, and threads. These features are closely related to the functionality and manufacturing process of the part.

[0037] (1) Meanwhile, the geometry of mechanical parts often contains a large amount of edge details, and calculating edge density can directly reflect the richness of contours and details in the projected sketch. This application uses an adaptive Canny edge detection algorithm for edge extraction and defines an edge density index. To quantify the complexity, as in equation (2), where, The number of pixels at the edge. This represents the total number of pixels. A higher edge density indicates richer geometric details in the projected sketch.

[0038] (2) Furthermore, the outline of the projected sketch reflects the overall geometric shape and topological structure, serving as an important clue for identification and retrieval. Mechanical parts often possess complex and prominent outline features, necessitating further outline analysis. This involves defining the outline quantization values ​​of the projected sketch. As shown in equation (3), where It is normalized roundness. It is the normalized aspect ratio. It is normalized convexity. First, nonlocal mean denoising is performed on the image, then the largest contour is extracted, and the three basic indicators of the contour—roundness, aspect ratio, and convexity—are calculated. Then, normalization and nonlinear processing are performed.

[0039] (3) Finally, the overall complexity of each projection sketch is calculated based on the three features obtained: information entropy value, edge density value, and contour quantization value, as shown in Equation (4). After multiple experiments, the weights in this application are set to 0.35, 0.3, and 0.35. The projection sketch with the highest complexity is selected as the initial projection sketch of the candidate set.

[0040] (4) Based on the initial projection sketch, a method to enhance the degree of difference is proposed. The remaining projection sketches are selected using the following criteria, as shown in Equation (5). Specifically, the three features of each projection sketch are combined into a feature vector, which is then standardized and L2 normalized. Finally, the cosine difference between the remaining projection sketches and all candidate projection sketches is calculated. The smaller the cosine similarity, the closer the similarity between the projected sketches. The larger it is. Then, normalized information entropy is introduced. As an enhancing factor, to balance information content and dissimilarity, we obtain ;choose The largest projection sketch is added to the candidate projection sketch set, and so on, until six projection sketches are selected.

[0041] (5) Using this method, this application is able to select a representative set of projection sketches that have a lot of information, high complexity, and sufficient diversity, thereby accurately and efficiently representing complex 3D models of parts.

[0042] In some embodiments, regarding step S2: Unlike ordinary 3D models, part 3D models often possess key local information such as holes, grooves, and threads, and lack visual effects, making the identification and differentiation of part 3D models quite difficult. To effectively extract features from part 3D models, this application introduces the idea of ​​dual-layer hypergraph deep learning and proposes a feature extraction method for part 3D models. This method constructs a residual network with an attention mechanism to extract global and key features from a single projected sketch, and constructs a dual-layer hypergraph neural network (DLHGNN) to extract and fuse features from multiple projected sketches of the part 3D model, finally obtaining a deep feature representation of the part 3D model.

[0043] First, feature extraction of the projected sketch is performed, and a residual network structure with a fusion attention mechanism for sketch feature extraction is constructed as follows: Figure 3 As shown, the input projection sketch undergoes preprocessing, including image resizing and normalization, to ensure data consistency and efficient network training. The preprocessed image is then processed through the initial convolutional layers of the ResNet18 network for preliminary feature extraction, and max pooling is used to reduce the dimensionality of the feature image, thereby reducing computational cost. In the deep feature extraction stage, the image is processed through multiple residual blocks, each containing a convolutional layer, batch normalization, and a ReLU activation function. This effectively solves the gradient vanishing problem in deep networks, improves the network's expressive power, and yields better global feature representation of the projection sketch. However, relying solely on the global information of the image is insufficient to meet the high-precision requirements of 3D part model retrieval. Unlike ordinary 3D models such as humans, animals, and buildings, which possess independent integrity, parts are only a portion of the overall assembly structure and are incomplete. The projection contours of individual parts are often not clear enough, making independent recognition from the assembly difficult. Therefore, it is more necessary to focus on their local key information. To address this, a lightweight attention module, CBAM (Convolutional Block Attention Module), is introduced. Channel attention enhances the channel response of key areas such as holes and threads, while spatial attention locates easily overlapping local key feature areas such as chamfered edges. The channel attention and spatial attention mechanisms are then cascaded to further improve the model's attention to local details of the projected sketch, enhance the discriminative power of the projected sketch features, and ultimately extract both global and local key features of the projected sketch.

[0044] However, single-view projection sketch features cannot comprehensively represent the complete information of a part's 3D model, especially from certain perspectives. For example, when 2D sketches of the same part's 3D model have significant visual differences, or when 2D sketches of dissimilar 3D models have significant visual similarities, representing the part's 3D model using a single-view projection sketch becomes even more difficult. Therefore, feature fusion of multiple selected projection sketches is crucial. Feature fusion not only integrates features from different projection sketches but also further extracts complex latent relationships between multiple perspective projection sketches, thereby improving the retrieval model's ability to understand the part as a whole.

[0045] Secondly, feature fusion of the 3D model of the part: a hypergraph is a structure capable of representing multi-dimensional relationships. It connects multiple nodes together through hyperedges, enabling the capture of more complex multi-dimensional relationships. Compared to ordinary graphs, hypergraphs can connect multiple nodes in a single edge, making them particularly suitable for representing the relationships between multiple projected sketches.

[0046] Therefore, this study introduces a two-layer hypergraph neural network (DLHGNN), such as Figure 4 As shown, features from different projection sketches are fused to obtain a deep feature representation of the part's 3D model. Figure 4 In the middle, the definition This represents the total number of part models. The total number of parts by category. ~ Indicates passage Figure 3 All projected sketch features of all extracted part models ~ Represents the features of the part model. ~ This represents the characteristics of m part categories.

[0047] In the first layer of hypergraph learning, the projected sketch features of all part models are... ~ As data input; construct hyperedges for multiple projected sketch features of the same part model, thereby constructing... Each hyperedge is aggregated into the hyperedge features of the first-layer hypergraph through hypergraph convolution, which are the features of the part model. ~ In the second layer of hypergraph learning, the obtained part model features are then... ~ As input, category hyperedges are constructed for each part model; the features of each part model are first propagated to the hyperedges through hyperedge aggregation to obtain the part category features. ~ Then, after feature update operations, the features of each part model are updated back. ~ The following is the specific implementation process.

[0048] The first-level hypergraph is a view-level hypergraph used to fuse features from various projected sketches of the same part model, compensate for structural incompleteness in the part representation, and obtain a preliminary feature representation of the part model.

[0049] definition This represents the part model number. When constructing the first-level hypergraph, all projected sketch features of each part model are used as nodes, and a hyperedge is generated by connecting them. A total of [number] parts were generated. An independent hyperedge, using hyperedge vectors Corresponding representation. For each part sample , This is used to indicate that all 6 projected sketch nodes of the same model are connected to a unique hyperedge. The element value of 1 in the matrix indicates that the connection exists, ensuring that all projected sketches of the same part model can exchange information with each other and achieve complete feature fusion. The hyperedge vectors of each part model constitute the first-level hypergraph matrix. .

[0050] definition , representing the 6 projection sketch numbers of the part model. Indicates passage Figure 3 All projected sketch features of all obtained part models ~ The set, Represents the 6 projected sketch features of the i-th part model ~ gather, Then it means the first The first part Each projection sketch feature.

[0051] In the first-layer hypergraph feature fusion process, the importance of different projected sketches is unequal, and importance is represented by weight values. Therefore, it is necessary to calculate the weight of each node. This application dynamically calculates the weight based on the distance between each projected sketch feature and the center of all projected sketch features of the part. Projected sketch features closer to the center are assigned higher weights, thus emphasizing their importance. The weights of all nodes are calculated accordingly. : (6) in, This represents the feature center of the projected sketch of the part model. The weight matrix of all hyperedges can be obtained from the node weights. The weight matrix of the first-level hypergraph can be obtained from all the hyperedge weight matrices. Therefore, node weighting is performed to obtain the weighted node features. : (7) Subsequently, a hypergraph convolution operation is performed on the first layer of the hypergraph neural network through a hypergraph convolution layer. The six projected sketch features of each model are weighted and propagated. The fusion of multiple projected sketch features is achieved through the node-hyperedge information propagation mechanism, generating a set of all hyperedge features containing higher-order relationships. That is, the feature set of the part model. The hypergraph convolution operation of the first layer hypergraph neural network is defined by equation (8), where, and These are the node degree matrix and hyperedge degree matrix of the first-level hypergraph, respectively. and These are their inverse matrices, used for normalization operations.

[0052] (8) The first-layer hypergraph performs weighted aggregation of the embedded representations of the projected sketches through hypergraph convolution, effectively integrating the complementary information between the various projected sketches of the part model and forming the preliminary features of each part's 3D model.

[0053] However, preliminary features alone are insufficient to accurately distinguish similar 3D part models across different categories. Therefore, a second-layer category-level hypergraph is constructed, injecting engineering semantics of part categories to further enhance the feature representation of the part models. In the construction of the second-layer hypergraph, the 3D part model features generated in the first layer are used as nodes, and hyperedges are constructed according to category labels to establish global connections between different part models of the same category. The expressive power of the part model features is further enhanced through two steps: feature fusion and feature update.

[0054] definition Indicates the part model category number, This represents the number of nodes for the k-th type of part. This represents the hyperedge matrix of the k-th type of part. 1 indicates that the feature of the part model is added as a node to the hyperedge matrix of this class. The number of nodes is determined by the actual number of part models in each category. The hyperedge matrices of each category constitute the second-level hypergraph matrix. .

[0055] The next step is to calculate the weight of each node, in total. If there are 1 part model, then there are Each node has its own weight. In the second-layer hypergraph, each hyperedge represents a part category, and the node weights are dynamically represented by calculating the distance from each part's model features to the feature center of that category. This allows us to obtain the weight value of each node, the weight matrix of each hyperedge, and the final weight matrix of the second-layer hypergraph. .

[0056] Subsequently, feature weighting operations are performed. The weighted features are propagated through the second hypergraph convolutional layer. During hypergraph propagation, features of similar part models shrink towards the class center, while features of dissimilar part models separate along the gradient direction of the hyperedge. The optimized features exhibit higher intra-class similarity and inter-class discriminability.

[0057] The second-layer hypergraph convolution operation includes two steps: feature aggregation and feature update. First, the features of the part model nodes are weighted and aggregated into the hyperedge of the part category to obtain the hyperedge features of the part category. Then, the hyperedge features of the part category are backpropagated to the part model nodes to update the node features of the part model. Therefore, the hypergraph convolution operation of the second-layer hypergraph neural network is defined by equation (9), where, and These are the node degree matrix and hyperedge degree matrix of the second-level hypergraph, respectively. It is the square root inverse of the degree matrix. It is the inverse of the hyperedge degree matrix, used for symmetric normalization operations to achieve fair information propagation and numerically stable computation. Finally, it is obtained through two steps: feature aggregation and feature update via hypergraph convolution. This refers to the updated feature set of all part models.

[0058] (9) The core objective of the second-layer hypergraph is to enhance the semantic discriminativeness of the feature space, achieving dual optimizations: first, reducing the distance between similar part models and increasing the density of features within a class; second, increasing the distance between different categories of part models, expanding the feature differences between different categories, and strengthening inter-class boundaries.

[0059] Through these two layers of hypergraph network learning, sketch features from different perspectives can be deeply integrated, and semantic enhancement of part model features in a category-oriented manner can be effectively achieved through feature updates, providing high-quality 3D model feature representations of parts for subsequent metric learning modules.

[0060] In some embodiments, regarding step S3: the obtained part model features are an overall feature representation that aggregates features from multiple projected sketches, while the features extracted from the user-input sketch are single two-dimensional image features. To eliminate the inherent significant inter-domain differences between the two and achieve accurate and intelligent matching between the sketch and the part's 3D model, a cross-domain matching method for 3D models based on multiple loss metric learning is proposed, such as... Figure 5 The core of this method lies in constructing a depth metric space that can accurately measure the similarity between sketches and 3D models of parts through the collaborative optimization of multiple loss functions.

[0061] First, a bi-branch adaptive projection network is used to process sketch features and part model features. Each branch consists of two fully connected layers, with batch normalization and ReLU activation functions in between. The outputs of the two branches are processed through a shared batch normalization layer to ensure the consistency of the distribution of the two types of features across different modalities. Finally, L2 normalization is performed to obtain a unit vector. This architecture preserves the modal features of both the sketch and the part model while creating a unified and comparable feature space, thus mitigating the problem of cross-modal domain differences.

[0062] To comprehensively optimize the feature space structure, this application proposes a multi-objective collaborative metric learning strategy by integrating three complementary loss functions. This strategy maps features from different modalities to a unified metric space, ensuring that semantically similar samples are closer together and semantically different samples are farther apart. During network training, a progressive training mechanism is constructed, comprising three stages: global initialization, local optimization, and fine-tuning. The scheduler automatically calculates the optimal weights based on the current training cycle, ensuring a smooth transition of the model from global optimization to local fine-tuning learning.

[0063] During the global initialization phase, temperature scaling is used to compare the loss. Establish the basic feature distribution. Construct a cross-modal global similarity matrix using this function, based on temperature parameters. The model's sensitivity to difficult samples is adjusted. The core objective is to calculate the cosine similarity between sketch features and all part models, then convert it to a probability distribution using a softmax function. Positive samples are sketch-model pairs of the same type and instance; the loss function forces the similarity of positive sample pairs to be higher than the weighted average of all negative sample pairs. This function design enables the model to effectively learn the global feature distribution structure in a large-scale negative sample environment, laying the foundation for subsequent optimization.

[0064] In the local optimization phase, an improved triplet loss weight is employed to achieve fine-grained instance differentiation. Traditional triplet losses primarily focus on establishing relative distance relationships, but in individual-level retrieval tasks, more refined constraint mechanisms are needed. The improved triplet loss designed in this application not only considers category similarity but also introduces individual-level constraints. For a given anchor point sample… (Sketch features) The corresponding part model features are used as positive samples. Other part models are used as negative samples. Improved triplet loss function Defined as equation (10), where, Represents Euclidean distance. N is the set boundary value, and N is the batch size.

[0065] (10) During the fine-tuning phase, the weights of the improved triplet loss are increased, the decision boundary is optimized, and hard sample processing is performed. Through online hard sample mining techniques, the most similar negative samples are automatically selected for training, significantly improving the model's ability to distinguish similar parts. This design enables the model to learn more refined feature representations, distinguishing not only samples from different categories but also different individual instances within the same category.

[0066] Cross-modal center loss during the entire training process Regular expressions are used to ensure compactness within a class.

[0067] During the retrieval phase, this application employs a hybrid matching strategy combining category-level matching and instance-level matching: category-level matching evaluates the deviation of sketch features from class centers, while instance-level matching directly compares feature vectors using cosine similarity. In tasks involving fine-grained differentiation between different instances of the same class and similar instances of different classes, this dual matching strategy significantly improves the retrieval accuracy of boundary samples.

[0068] In some embodiments, regarding step S4: To enhance information transfer among the projection sketch feature extraction module, the two-layer hypergraph feature fusion module, and the multiple loss metric learning module, enabling each module network to collaboratively adjust its learning direction according to the needs of the retrieval task, thereby significantly improving overall retrieval performance and overcoming the problem of high time and performance overhead in traditional modular individual training methods, this application proposes an end-to-end training framework for 3D model retrieval of parts based on a joint loss function, such as... Figure 6 As shown, its principle lies in constructing a unified and optimized overall training system that seamlessly connects the three major modules of feature extraction, two-layer hypergraph feature fusion, and multiple loss metric learning to form a closed-loop optimization mechanism. Through deep neural networks, it achieves full-process integration from raw data input to retrieval result output.

[0069] At the implementation level, the framework employs a dynamic gradient propagation mechanism to achieve collaborative parameter updates. That is, after the joint loss function is calculated, the gradient signal is simultaneously transmitted to all sub-modules through the backpropagation algorithm: the gradient calculation of the feature extraction network not only depends on the classification loss but is also directly guided by the metric learning loss, ensuring that its learning direction always remains consistent with the retrieval target; the feature fusion module receives dual supervision from the classification task and feature alignment, dynamically adjusting the feature aggregation strategy; and the projection network parameters of the metric learning module are optimized in real time according to the cross-modal alignment requirements.

[0070] In terms of training strategy, the proposed end-to-end training framework dynamically adjusts the classification loss weights. and metric loss weights A three-stage dynamic optimization strategy is constructed: in the early training phase (0-30% of the time), classification loss is the primary method. =0.7), focusing on building basic feature representation capabilities; in the medium term (30-70% of the cycle), balancing classification and measurement loss ( =0.5, =0.5), achieving synergistic optimization of feature discriminativeness and modality alignment; in the later stages (70-100% of cycles), the focus is on measuring loss ( =0.7), focusing on fine-tuning of cross-modal feature space.

[0071] The core function of the end-to-end training architecture for 3D model retrieval of parts based on joint loss function lies in overcoming the limitations of traditional staged training. On the one hand, it solves the problem of inconsistent objectives among modules, where each module network no longer optimizes the classification task in isolation, but directly serves the final retrieval objective. On the other hand, end-to-end gradient propagation eliminates feature space discontinuities in traditional methods, enabling unified feature representation across modalities.

[0072] This application also provides practical examples based on the above methods: This example uses the ESB (Engineering Shape Benchmark) dataset for model training and experimental evaluation. The ESB dataset contains 867 mechanical part models across 42 categories, with 66 models falling into three miscellaneous categories. Since miscellaneous models are not further subdivided and would interfere with model training, they were ignored in this application's training. Additionally, this application downloaded 254 part models from the TraceParts and 3DSource parts libraries as supplementary data, expanding the dataset to a total of 1055 part models.

[0073] To fully validate the performance of the proposed retrieval model, six evaluation metrics widely used in the field of three-dimensional model retrieval were employed for evaluation, and the retrieval performance was further evaluated intuitively through the recall-precision (PR) curve.

[0074] (1) Nearest neighbor (NN): represents the average precision of the first most accurate result retrieved; (2) First tier (FT): Represents the recall rate of the top k results in the search results; (3) Second tier (ST): Represents the recall rate of the top 2,000 results in the search results; (4) E-measure (E): A comprehensive metric obtained by solving the harmonic mean of recall and precision, which is used to comprehensively evaluate recall and precision; (5) Discounted cumulative gain (DCG): An indicator that comprehensively considers the relevance and ranking quality of search results. It assigns weights to search results using relevance as a positive proportional coefficient and discounts the weights to obtain the cumulative gain value. (6) Mean average precision (mAP): A retrieval evaluation metric used for multiple queries. It is obtained by averaging the average precision scores of all queries and reflects the accuracy and stability of the system for different queries.

[0075] The entire end-to-end training network presented in this example is implemented in PyTorch, and all experiments are run on a server equipped with an NVIDIA RTX A5000 with 24GB of video memory and an Intel(R) Xeon(R) Platinum 8358P CPU.

[0076] In the data preprocessing stage, firstly, the MVCNN 3D model rendering method is used to obtain 12 views at 30° intervals; then, a sketching operation is performed to obtain 12 projected sketches of the part model, and then 6 projected sketches are selected to represent the 3D model of the part; subsequently, the size of the obtained part model projected sketches and hand-drawn sketches is uniformly set to 224×224×3, which are used as inputs for end-to-end training network.

[0077] For the experimental parameter settings, in the end-to-end network training, the dataset was divided into training, validation, and test sets in a 6:2:2 ratio. The training batch size was set to 32, and a gradient accumulation mechanism (accumulation_steps=2) was used to simulate an effective batch size of 64. The initial learning rate was set to 3×10⁻⁻⁴. 5 The network was tuned using a cosine annealing warp restart scheduler, with a maximum training epoch of 150, an early stopping patience of 20, a weight decay of 0.01, and a gradient pruning threshold of 1.0. The classification loss weights and metric loss weights were dynamically adjusted to optimize the network parameters.

[0078] Experiments were conducted using the expanded ESB dataset to test the proposed method, and a comparative analysis was performed with seven existing commonly used sketch-based 3D model retrieval methods, including cross-domain manifold ranking (CDMR), sketch-based 3D model retrieval based on view clustering (SBR-VC), the Siamese method based on binary metric learning, learned Wasserstein barycentric representation (LWBR), deep point-to-subspace metric learning (DPSML), sketch feature learning with uncertainty (SUL), and sketched 3D part retrieval method integrating dual scale features (IDSF). The comparison results are as follows. Figure 9 As shown, the PR curve results are as follows: Figure 7 As shown.

[0079] Depend on Figure 9 As can be seen, compared with other methods, the method proposed in this application shows certain improvements in various indicators, especially an improvement of NN by 0.181 and mAP by 0.018. Figure 7 As can be seen, the method in this application outperforms other methods on the PR curve, thus demonstrating a significant advantage. Current commonly used methods either utilize dual-network models to extract view and sketch features separately, or seek the optimal perspective to improve cross-domain learning capabilities. This application addresses the redundancy problem of multi-view information, focuses on extracting key local features to compensate for deficiencies in global feature representation, and employs a feature fusion mechanism using a two-layer hypergraph neural network to further enhance the feature semantics of the part model, thus exhibiting better performance.

[0080] To verify the effectiveness of the projection sketch representation and filtering method proposed in this application, four experimental datasets of different sizes were constructed by controlling the maximum threshold for the number of projection sketches to be filtered. These included: 12 unfiltered projection sketches, and datasets containing 3, 6, and 9 projection sketches for each part model, respectively. Figure 10 The results are based on datasets of different numbers of projected sketches.

[0081] from Figure 10 As can be seen, the configuration with 6 projection sketches yields the best retrieval results, while the performance of the other three configurations is poor. The main reasons are as follows: When the number of projection sketches is too small, it is impossible to fully capture the complete geometric information and multi-view features of the part, thus failing to comprehensively represent the 3D model of the part; while the dataset using 12 unfiltered projection sketches contains more information, it contains a large amount of similar or redundant view information, which interferes with the overall feature representation. The dataset with 6 filtered projection sketches can balance the completeness and redundancy of view information, retaining enough projection sketches to comprehensively represent the 3D model of the part while removing redundant projection sketches, enabling the network to learn more compact and discriminative feature representations, thereby improving retrieval performance.

[0082] To verify the effectiveness of the network structure of the two-layer hypergraph feature extraction model, four different network architectures were designed for ablation experiments. The first architecture is ResNet18+DLHGNN-Layer1, which uses a basic ResNet18 network for feature extraction, then connects to the first layer of the two-layer hypergraph network for feature aggregation. The second architecture is ResNet18+CBAM+DLHGNN-Layer1, which adds the CBAM attention mechanism to ResNet18 to enhance feature extraction capabilities, again using only the first layer of the two-layer hypergraph network to verify the contribution of CBAM. The third architecture is ResNet18+DLHGNN, which uses a complete two-layer hypergraph network to replace the single-layer structure in the first method, verifying the advantages of the two-layer hypergraph network compared to the single-layer structure. The fourth architecture is ResNet18+CBAM+DLHGNN, the complete network architecture proposed in this application, combining a CBAM-enhanced ResNet18 feature extraction module with a complete two-layer hypergraph network. Figure 11 The retrieval performance under different network structures is demonstrated.

[0083] As shown in the table, comparing the results of the first and second network architectures reveals that, under the same single-layer hypergraph network structure, adding the CBAM module significantly improves the mAP value, indicating that the CBAM attention mechanism effectively enhances feature extraction capabilities and improves the quality of sketch representation. Comparing the first and third network architectures shows that using the complete two-layer hypergraph network achieves better retrieval results than using only a single layer, confirming the superiority of the two-layer hypergraph network in feature fusion. The final complete architecture combines the advantages of two key modules, achieving the highest mAP value compared to the other three simplified versions, further improving the overall feature representation capability and retrieval accuracy of the part model.

[0084] To further verify the effectiveness of the proposed feature extraction model, the obtained fused features were visualized using the t-SNE feature dimensionality reduction method. The scatter plots of the features for the 30 categories of part models are shown below. Figure 8 As shown in the scatter plot, it is clear that different categories of 3D part models form obvious clusters in the feature space, with clear boundaries between categories and no serious category mixing, indicating that the feature extraction model can effectively distinguish different types of part models. Part model samples within the same category are tightly clustered in the feature space, indicating that the feature extraction model can capture the common features of similar parts and has good intra-class consistency. The scatter plot experiment intuitively proves that the two-layer hypergraph feature extraction model can effectively learn a highly discriminative 3D part model feature representation by fusing sketch features.

[0085] To evaluate the end-to-end trained network and its loss function, ablation experiments were conducted to compare the retrieval results using only individual network modules trained and optimized with their respective loss functions, with the retrieval results using end-to-end network training and joint loss function optimization. Figure 12 As shown, using end-to-end network training and joint loss function can achieve better retrieval results.

[0086] Figure 13 The image shows an example of the search results for the ESB dataset. The second column contains several randomly selected hand-drawn sketches, and the third column contains the first nine 3D models of the retrieved parts.

[0087] As can be seen from the table, the method described in this application can obtain highly accurate retrieval results for these 3D part models. Furthermore, due to the feature fusion and update mechanism of the two-layer hypergraph network, the retrieval system also demonstrates strong discriminative ability when retrieving easily confused similar part models, especially for different individuals of the same type of part, and for different categories of parts with similar overall features. Since the features of these two types of models are often highly similar, the retrieval system is easily confused. Specific case analyses are as follows.

[0088] exist Figure 13 In the gear retrieval example, when a hand-drawn sketch with tooth profiles, a central hub hole, and surrounding small holes is input, the retrieval system can accurately match the target model from multiple similar gears, indicating its ability to accurately identify key local features such as micro-holes. In the bolt retrieval, when a hand-drawn sketch with cross-shaped grooves is input, the top three search results are all bolts with cross-shaped grooves, differing only in the head-shank size ratio. The bolt with the closest ratio to the retrieved sketch ranks highest, demonstrating the system's ability to effectively capture geometric proportions. These two examples demonstrate the retrieval system's ability to distinguish between different parts of the same type.

[0089] On the other hand, Figure 13 In the example of the pipe bend search, the first half of the search results list is related to the search results for the pipe bend. Figure 1 The simple bends in the first half are replaced by more complex 90-degree bends in the second half. The ranking results clearly demonstrate the system's ability to distinguish between similar models of different categories. Similarly, in the bearing category, although the first result (bearing) and the third result (flange part) are highly similar in their main geometric shape, both consisting of two parts with a smaller upper part and a larger lower part with basically the same proportion, the feature extraction model can accurately extract the small hole features of the base of the flange part and distinguish them. This further verifies the system's ability to identify highly similar models of different categories.

[0090] In summary, the innovative points of this application are: 1. By integrating complexity and diversity, a new method for selecting projection sketches is proposed, which can effectively filter out a representative set of projection sketches that have more information, higher complexity, and sufficient diversity for the semantic representation of the 3D model of a part. 2. A two-layer hypergraph neural network is introduced to extract and fuse features from multiple projection sketches of the 3D model of the part, realizing deep fusion of sketch features from different perspectives and category-oriented semantic enhancement; 3. A cross-domain matching method for 3D part models based on multiple loss metric learning was proposed. A deep metric space that can accurately measure the similarity between hand-drawn sketches and 3D part models was constructed. An end-to-end training framework for 3D part model retrieval based on joint loss function was developed to achieve accurate and efficient matching between sketches and 3D part models. 4. Experiments were conducted on the ESB dataset, and the results were quantified and evaluated using multiple metrics. The experimental results show that, compared with some other commonly used sketch retrieval methods, the method in this application has better retrieval performance and can effectively improve the retrieval results of 3D models of parts.

[0091] The method described in this application has already shown good results on the ESB dataset. Future experiments and possible method improvements will be conducted on general datasets such as Shrec2013 and Shrec2014 to further enhance the universality and generalization ability of the research results.

[0092] Furthermore, this application uses a fully connected approach to connect the six projected sketches of the part model during the hypergraph network construction process. Features of projected sketches from important angles may be diluted by features from less important projected sketches, thus affecting the discriminative features of specific combinations of projected sketches. Therefore, subsequent iterations will further extract similar viewpoints for multi-hyperedge construction. During training, the projected sketch combinations of key viewpoints will be learned, ensuring that low-quality projected sketches only affect their respective hyperedges without interfering with the key features of other hyperedges. This enhances feature complementarity and further improves retrieval accuracy.

[0093] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that the requirements of this application encompass various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this application and are within the spirit and scope of the exemplary embodiments of this application.

[0094] Furthermore, it should be understood that in the foregoing description of the embodiments of this application, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this application. That is, the embodiments in this application can also be understood as an integration of multiple sub-embodiments. It is also valid when each sub-embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0095] Finally, it should be understood that the embodiments disclosed in this application are illustrative of the principles of the embodiments of this application. Other modified embodiments are also within the scope of this application. Therefore, the embodiments disclosed in this application are merely examples and not limitations. Those skilled in the art can adopt alternative configurations to implement the applications in this application based on the embodiments in this application. Therefore, the embodiments of this application are not limited to the embodiments precisely described in the application.

Claims

1. A cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion, characterized in that, Includes the following steps: S1: Projection sketch generation and selection: Multiple projection views are obtained by multi-angle projection of the 3D model of the part. Each projection view is converted into a corresponding projection sketch. Complexity analysis and difference calculation are performed on the projection sketches. Iterative selection is carried out to obtain a set of projection sketches representing the 3D model of the part. S2: Feature extraction of the 3D model of the part. A deep residual network with a fusion attention mechanism is used to extract the global features and local key features of each projection sketch in the projection sketch set. Then, a two-layer hypergraph neural network is used for feature extraction and fusion to obtain the 3D model features of the part with deep fusion and semantic enhancement. S3: Cross-domain feature matching. Construct a metric learning model based on multiple losses to map the features of hand-drawn sketches and the features of the 3D model of the part to the same metric space to achieve cross-domain feature alignment. Calculate similarity based on the aligned features to complete cross-domain matching. S4: End-to-end training and retrieval. An end-to-end training framework with a joint loss function is built to connect the projection sketch generation and selection, part 3D model feature extraction and cross-domain feature matching operations. The joint loss function dynamically and collaboratively optimizes the parameters of each operation and outputs the retrieval results of the target part 3D model.

2. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 1, characterized in that, In step S1, each of the projected views is converted into a corresponding projected sketch, specifically including: Each of the projected views is converted into a corresponding projected sketch by incorporating an edge detection algorithm into the sketching process. The edge detection algorithms include the Sobel edge detection algorithm and the Canny edge detection algorithm; The sketching process involves: extracting the contour edges of the projected view using the edge detection algorithm, performing a line-based transformation on the contour edges, and generating the projected sketch that conforms to the hand-drawn sketch modality.

3. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 1, characterized in that, In step S1, complexity analysis and difference calculation are performed on the projected sketch, specifically including: Complexity analysis: Construct a complexity evaluation system that includes information entropy, edge density, and contour features, and calculate the complexity value of each projected sketch; select the projected sketch with the highest overall complexity as the initial candidate sketch; Difference Calculation: For the initial candidate sketch and other projection sketches to be screened, feature vectors are constructed based on the information entropy, edge density, and contour features, respectively. The cosine similarity between the two types of feature vectors is calculated to evaluate the difference between the projection sketches. The information entropy of the projection sketch is introduced and normalized. The normalized information entropy is used as the information content index to correct the difference evaluation results. The selection is iteratively carried out until a set of projection sketches of a preset size is obtained.

4. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 3, characterized in that, The process of obtaining the contour features is as follows: first, nonlocal mean denoising is performed on the projected sketch to eliminate noise interference, and then the main contour of the denoised projected sketch is extracted; the geometric parameters of the main contour are calculated, including roundness, aspect ratio and convexity, and the geometric parameters are normalized and nonlinearly processed respectively to obtain the contour features used for complexity analysis.

5. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 1, characterized in that, In step S2, the deep residual network with fused attention mechanism includes: Using ResNet series deep residual networks as the base network, a convolutional block attention module is integrated into the feature extraction link of the base network; The convolutional block attention module includes a channel attention submodule and a spatial attention submodule, which are cascaded together. The channel attention submodule enhances the feature channel response corresponding to the key structure of the part's 3D model by analyzing the importance of each feature channel; the key structure is a special geometric feature on the 3D model. The spatial attention submodule captures local detail information by locating the spatial position of local key features in the projection sketch, thereby achieving simultaneous extraction of global features and local key features.

6. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 1, characterized in that, In step S2, the two-layer hypergraph neural network performs feature extraction and fusion, including: View-level hypergraph processing: Using the combined features of global and local key features of each projection sketch corresponding to the 3D model of the same part as nodes, a hyperedge is constructed for each 3D model of the part according to the part's affiliation; the node weight of each node is dynamically calculated based on the distance from each node's features to the feature center of the corresponding 3D model of the part; and all node features are aggregated through hypergraph convolution operations to obtain the preliminary features of the 3D model of the part. Category-level hypergraph processing: Using the preliminary features of multiple 3D models of the parts as nodes, construct hyperedges for each part category according to the part category label; dynamically calculate the node weight of each node based on the distance from the feature of each node to the category feature center of the corresponding part category; perform feature aggregation and feature update operations sequentially through hypergraph convolution, and output the 3D model features of the parts with high intra-class feature compactness and high inter-class feature discriminability.

7. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 6, characterized in that, The hypergraph convolution operation includes: Node weighting: The features of the corresponding nodes are weighted according to the node weights to obtain weighted node features; Hyperedge information transmission: The features of all weighted nodes connected by the same hyperedge are transmitted to that hyperedge, realizing the interaction of node feature information within the hyperedge range; Feature normalization: Introduce the node degree matrix and hyperedge degree matrix of the hypergraph, calculate the inverse matrix of the node degree matrix and hyperedge degree matrix, and normalize the features after information transmission using the inverse matrix to ensure the fairness and numerical stability of information transmission within the hypergraph.

8. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 1, characterized in that, In step S3, the metric learning model for multiple losses includes: Two-branch feature projection: Construct a two-branch adaptive projection network, where one branch is used to process the features of the hand-drawn sketch and the other branch is used to process the features of the 3D model of the part; perform feature mapping on the two types of features through the two-branch adaptive projection network, and perform normalization processing on the mapped features to obtain the standardized feature vector in the same metric space. Multiple loss construction: The multiple loss is formed by fusing contrastive loss, triplet loss and cross-domain center loss; a progressive training mechanism is constructed based on the multiple loss to optimize the metric space, so that semantically similar features are closer in the metric space and semantically different features are farther apart in the metric space. Hybrid matching: The retrieval stage employs a strategy that combines category-level matching and instance-level matching; the category-level matching achieves coarse cross-domain matching by evaluating the deviation between the standardized feature vector and the category feature center of the corresponding part category; the instance-level matching achieves fine cross-domain matching by calculating the cosine similarity between the standardized feature vectors.

9. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 8, characterized in that, The progressive training mechanism comprises three sequentially connected stages: Global initialization phase: Based on the contrastive loss, construct the basic feature distribution of the metric space to initially separate features of different categories in the metric space; Local optimization phase: Based on the triplet loss, constraints are applied to the features of different instances of the same type to improve the ability to distinguish instance-level features; The features include: features of 3D models of different parts of the same type, features of hand-drawn sketches and 3D models of non-target parts of the same type. Fine-tuning phase: Introducing online hard sample mining technology to select easily confused hard samples in the metric space to participate in training, optimizing feature decision boundaries, and further improving cross-domain matching accuracy.

10. The cross-domain retrieval method for sketched part 3D models based on dual-layer hypergraph feature fusion according to claim 1, characterized in that, In step S4, the joint loss function is an integration of classification loss and metric loss; The parameters of each module in the dynamic collaborative optimization include: Based on the training cycle ratio, the training cycle is divided into the early training period, the middle training period, and the late training period; In the early stages of training, the classification loss is the primary method used to build basic feature representation capabilities. During training, the weights of the classification loss and the metric loss are balanced to achieve synergistic optimization of feature discriminativeness and cross-domain alignment. In the later stages of training, the aforementioned metric loss is the primary focus, with an emphasis on fine-tuning across the feature space. Through the gradient backpropagation mechanism, the loss signal generated by the joint loss function is synchronously transmitted to the projection sketch generation and selection, part 3D model feature extraction and cross-domain feature matching modules, so as to realize the collaborative update of parameters of all modules.

Images