Functional part division method and device of three-dimensional model, electronic equipment and storage medium

Abstract

The invention relates to a functional part division method and device of a three-dimensional model, electronic equipment and a storage medium. The method comprises the steps of obtaining point cloud data and a multi-view two-dimensional image of the three-dimensional model; based on the surface ID of each curved surface in the three-dimensional model, binding the surface ID of the corresponding curved surface for each sampling point of the point cloud data, and binding the surface ID of the corresponding curved surface of the two-dimensional plane for each pixel of the multi-view two-dimensional image; performing cross-modal attention fusion on the point cloud data and the multi-view two-dimensional image through a preset semantic segmentation model by taking the surface ID as an association reference to obtain a fused point cloud feature, and determining a functional region label of each sampling point in the fused point cloud feature; mapping the function area label of each sampling point to the surface ID bound with the corresponding sampling point to obtain a function area label corresponding to each surface ID; and combining the surface IDs with the same functional area labels into a curved surface set. According to the invention, accurate identification of each functional part of the three-dimensional model is realized.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to a method, apparatus, electronic device, and storage medium for dividing functional parts of a three-dimensional model. Background Technology

[0002] Computer-aided engineering (CAE) analysis is a crucial part of the product development process. In the CAE analysis preprocessing stage, the geometric model imported from the computer-aided design (CAD) system needs to be grouped and named as components. Components with different structures need to use different mesh generation standards, which is the basis for ensuring the accuracy of CAE analysis.

[0003] Currently, the industry mainly relies on automated tools based on predefined rule scripts and simple string matching to group and name components in CAD geometric models. These methods rely on a single grouping criterion, classifying components only based on basic attributes such as material and dimensions. They cannot accurately distinguish between different functional parts, and therefore cannot set differentiated meshing standards for different structural characteristics. This deficiency leads to a mismatch between the mesh and the actual structure of the components, directly affecting the accuracy and efficiency of CAE analysis and failing to meet the actual needs of high-precision CAE analysis. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for dividing functional parts of a three-dimensional model to solve the problem of not being able to accurately distinguish functional parts of a three-dimensional model.

[0005] Firstly, this application provides a method for dividing a three-dimensional model into functional parts, the method comprising: The point cloud data and multi-view two-dimensional images of a three-dimensional model are acquired, wherein the point cloud data is used to indicate the local geometric details of each surface of the three-dimensional model, and the multi-view two-dimensional images are used to indicate the global spatial structure of the three-dimensional model. Based on the surface ID of each surface in the 3D model, each sampling point of the point cloud data is bound to the surface ID of the surface to which it belongs, and each pixel of the multi-view 2D image is bound to the surface ID of the surface corresponding to the 2D plane in which it is located. Using a pre-defined semantic segmentation model and with face ID as the association benchmark, the point cloud data and the multi-view 2D image are fused across modal attention to obtain fused point cloud features, and the functional region label of each sampling point in the fused point cloud features is determined. Map the functional area label of each sampling point to the face ID bound to the corresponding sampling point to obtain the functional area label corresponding to each face ID; Face IDs with the same functional area label are merged into a surface set, where each surface set is used to indicate a functional part.

[0006] Optionally, using a preset semantic segmentation model and with the face ID as the association criterion, the point cloud data and the multi-view 2D image are fused across modal attention to obtain fused point cloud features, and the functional region label of each sampling point in the fused point cloud features is determined to include: The point cloud data and the multi-view two-dimensional image are input into a preset semantic segmentation model; The image encoder in the semantic segmentation model performs feature aggregation on the multi-view two-dimensional image to obtain image features, and the point cloud encoder in the semantic segmentation model extracts features from the point cloud data to obtain point cloud features. The image features can capture the feature correlation of the same surface under multiple views, and the point cloud features can distinguish regions that are physically close but belong to different parts. The image features and the point cloud features are fused by a cross-modal attention layer, and the image features and point cloud features corresponding to the same face ID are associated during the fusion process to obtain fused point cloud features. The image encoder and the point cloud encoder share the cross-modal attention layer. The fused point cloud features are decoded and classified using a decoder to obtain the functional region label for each sampling point.

[0007] Optionally, the multi-view two-dimensional image is subjected to feature aggregation by the image encoder in the semantic segmentation model to obtain image features including: The image encoder maps the face ID bound to each pixel in the multi-view 2D image to a feature vector. The feature vectors of all pixels corresponding to the same face ID in a single two-dimensional image are aggregated in the spatial dimension to obtain the single-view feature vector of each face ID in the single two-dimensional image. The single-view feature vectors of all two-dimensional images for the same face ID are fused to obtain the fused feature vector for each face ID; The fused feature vectors of each face ID are concatenated to obtain the image features.

[0008] Optionally, the image features and the point cloud features are fused through a cross-modal attention layer, and the image features and point cloud features corresponding to the same face ID are associated during the fusion process to obtain fused point cloud features including: Feature channels of the image features and the point cloud features are unified through a cross-modal attention layer; Based on surface ID consistency, attention-weighted fusion is performed on the unified image features and point cloud features to obtain a first fused point cloud feature, wherein the first fused point cloud feature is used to indicate the preliminary association between the image features and the point cloud features. The residuals of the first fused point cloud feature and the point cloud feature after unifying the channel are added and normalized to obtain the second fused point cloud feature. The second fused point cloud feature is used to incorporate the image features and the correlation between the point cloud features while retaining the point cloud features. After the second fused point cloud feature is mapped by the feedforward network feature, the residuals of the second fused point cloud feature are added together and normalized to obtain the final fused point cloud feature.

[0009] Optionally, based on surface ID consistency, attention-weighted fusion is performed on the channel-unified image features and point cloud features to obtain the first fused point cloud features, including: A query vector is generated based on the point cloud features after unification, and a key vector and a value vector are generated based on the image features after unification. The query vector is used to indicate the features of each sampling point, the key vector is used to indicate the features of each pixel, and the value vector is used to indicate the feature value of each pixel. Traverse each sampling point and each pixel, and determine the bias term based on whether the face IDs bound to the sampling point and the pixel are consistent. The bias term corresponding to consistent face IDs is greater than the bias term corresponding to inconsistent face IDs. Attention weighting is performed based on the query vector, key vector, value vector and bias term to obtain the first fused point cloud feature.

[0010] Optionally, the fused point cloud features are decoded and classified using a decoder to obtain the functional region label for each sampling point, including: The decoder maps the features of each sampling point in the fused point cloud features to the dimension corresponding to each functional region label, thereby obtaining the score of each sampling point under each functional region label. The scores are normalized to obtain the predicted probability of each sampling point in the corresponding dimension of each functional area label; The functional region label corresponding to the highest predicted probability is selected as the functional region label of the sampling point.

[0011] Optionally, the training process of the semantic segmentation model includes: Acquire training data and input the training data into the initial semantic segmentation model, wherein the training data includes sample multi-view two-dimensional images, sample point cloud data, and actual functional region labels corresponding to each sampling point; The initial semantic segmentation model is used to process the multi-view two-dimensional images of the samples and the point cloud data of the samples, and output the predicted functional region labels of each sampling point. The loss between the predicted functional region label and the actual functional region label is calculated based on the joint loss function, wherein the joint loss function is constructed based on weighted cross-entropy loss, Dessian loss and in-plane consistency loss; During backpropagation, the parameters of the image encoder, the point cloud encoder, the cross-modal attention layer, and the decoder are updated together based on the loss signals of the image encoder and the point cloud encoder until the model converges, resulting in a trained semantic segmentation model.

[0012] Secondly, this application provides a functional part division device for a three-dimensional model, the device comprising: The acquisition module is used to acquire point cloud data and multi-view two-dimensional images of a three-dimensional model. The point cloud data is used to indicate the local geometric details of each surface of the three-dimensional model, and the multi-view two-dimensional images are used to indicate the global spatial structure of the three-dimensional model. The binding module is used to bind the surface ID of the surface to each sampling point of the point cloud data based on the surface ID of each surface in the 3D model, and to bind the surface ID of the surface corresponding to the 2D plane to each pixel of the multi-view 2D image. The fusion module is used to perform cross-modal attention fusion of the point cloud data and the multi-view two-dimensional image using a preset semantic segmentation model and with the face ID as the association reference, to obtain fused point cloud features and determine the functional region label of each sampling point in the fused point cloud features. The mapping module is used to map the functional area label of each sampling point to the face ID bound to the corresponding sampling point, so as to obtain the functional area label corresponding to each face ID; The assignment module is used to merge face IDs with the same functional area label into a surface set, where each surface set is used to indicate a functional part.

[0013] Thirdly, this application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0014] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for executing the functional part division method of the three-dimensional model described in any of the preceding claims of this application.

[0015] The technical solutions provided in this application have the following advantages compared with the prior art: In this application, point cloud data and multi-view 2D images of a 3D model are acquired. Each sampling point in the point cloud data and each pixel in the multi-view 2D images are then bound to a face ID of the 3D model. Using the face ID as the association, cross-modal attention fusion is performed on the point cloud data and multi-view 2D images corresponding to the same face ID. Leveraging the strong constraint of the face ID, this ensures that the same surface is not incorrectly segmented or confused with other surfaces. Simultaneously, cross-modal fusion considers both the local geometric details of each surface in the point cloud and the global spatial structure of the multi-view 2D images, resulting in fused point cloud features. Based on these fused point cloud features, functional region labels for each sampling point are obtained. By mapping the functional region label of each sampling point to the corresponding face ID bound to that sampling point, a one-to-one correspondence between face IDs and functional region labels is established. Face IDs with the same functional region label are then merged into a surface set, ultimately determining each functional part of the 3D model. This application achieves accurate identification of each functional part of the 3D model through cross-modal fusion constrained by face IDs and label mapping. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1 A schematic diagram of the hardware environment for naming 3D model parts provided in the embodiments of this application; Figure 2 A flowchart illustrating a method for dividing a three-dimensional model into functional parts, as provided in an embodiment of this application; Figure 3 A flowchart illustrating the semantic segmentation model provided in this application embodiment; Figure 4 This is a schematic diagram of the overall process provided for an embodiment of this application; Figure 5This application provides a schematic diagram of a functional parts division device for a three-dimensional model. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0022] To address the problem of inaccurate differentiation of functional parts in 3D models mentioned in the background art, this application embodiment uses surface ID as the link between cross-modal data and combines point cloud detail features with the global structure of multi-view 2D images for semantic segmentation to achieve accurate identification of functional parts.

[0023] The application scenarios of this application include, but are not limited to: grouping of 3D model parts, CAE simulation preprocessing, or complex assembly simulation preprocessing.

[0024] Optionally, in the embodiments of this application, the above-described method for dividing the functional parts of a three-dimensional model can be applied to, for example... Figure 1 The hardware environment shown consists of terminal 101 and server 103. Figure 1 As shown, server 103 connects to terminal 101 via a network. Terminal 101 sends the 3D model file and multi-view 2D images of the 3D model to server 103. Server 103 generates point cloud data of the 3D model, and then determines each functional part of the 3D model based on the point cloud data and multi-view 2D images, assigning a part identifier to each functional part. Database 105 can be set up on server 103 or independently of server 105 to provide data storage services for server 103. The aforementioned network includes, but is not limited to, wide area network, metropolitan area network, or local area network. Terminal 101 includes, but is not limited to, PC, mobile phone, tablet computer, etc.

[0025] The following will describe in detail a method for dividing a three-dimensional model into functional parts, based on specific implementation methods, taking an application to a server as an example. Figure 2 As shown, the specific steps are as follows: Step 201: Obtain point cloud data and multi-view 2D images of the 3D model. The point cloud data is used to indicate the local geometric details of each surface of the 3D model, and the multi-view 2D images are used to indicate the global spatial structure of the 3D model. Step 202: Based on the surface ID of each surface in the 3D model, bind the surface ID of the surface to each sampling point of the point cloud data, and bind the surface ID of the surface corresponding to the 2D plane to each pixel of the multi-view 2D image; Step 203: Using a preset semantic segmentation model and with the face ID as the association benchmark, perform cross-modal attention fusion between point cloud data and multi-view 2D images to obtain fused point cloud features, and determine the functional region label of each sampling point in the fused point cloud features. Step 204: Map the functional area label of each sampling point to the face ID bound to the corresponding sampling point to obtain the functional area label corresponding to each face ID; Step 205: Merge the face IDs with the same functional area label into a surface set, and assign part identifiers to the surface set, wherein each surface set is used to indicate a functional part.

[0026] In step 201, the terminal performs multi-view projection rendering on the 3D model based on the principal axis direction and the standard six-view perspective, generating multi-view 2D images. These multi-view 2D images, along with the 3D model file, are then sent to the server. The multi-view 2D images can fully present the visible areas and corresponding relationships of the same surface from different perspectives, clearly indicating the spatial positions of various surfaces in the 3D model, thereby accurately constructing the global spatial structure of the 3D model. For example, the same plane in the 3D model can fully display its outline and dimensions in the front view, while only its edge lines are shown in the side view. Through the differences in visibility and correspondence between multiple perspectives, the global spatial constraints of each surface in the overall assembly structure can be accurately captured.

[0027] The terminal performs multi-angle projection photography on the 3D model based on its main axis and six views, obtaining multi-view 2D images. These multi-view 2D images and the 3D model file are then sent to the server. In this embodiment, the 3D model file refers to a CAD file. Multi-view 2D images can present the visible area of ​​the same surface from different perspectives, thus reflecting the spatial distribution relationship between surfaces. Simultaneously, multi-view 2D images can characterize the relative positions of surfaces, reflecting their adjacency, subordination, and enclosure relationships. Furthermore, they can capture viewpoint occlusion information, demonstrating the visibility and occlusion status of each surface from different observation angles. Therefore, the spatial association of surfaces, their relative positional relationships, and viewpoint occlusion information of the 3D model can be used as the global spatial structure of the 3D model.

[0028] For example, the same plane in a 3D model can fully display its outline and size in the front view, but only its edge lines can be displayed in the side view. By observing the differences and correspondences of the visible range of the same surface under multiple perspectives, the global spatial relationship of each face of the 3D model can be accurately captured, thereby reflecting the global spatial structure of the 3D model.

[0029] In this application, "surface" is a general term for various geometric surfaces of a three-dimensional model, including both planar and curved surfaces. Planar surfaces include, for example, the plane of a car door anti-collision beam or the plane of a chassis reinforcement plate; curved surfaces include, for example, the surface of a car body panel with streamlined curvature, such as the outer door panel or the hood.

[0030] After receiving the 3D model file from the terminal, the server discretizes the file to generate corresponding point cloud data. This point cloud data consists of a large number of discrete sampling points, each with unique 3D spatial coordinates. Sampling points sharing the same surface may not be spatially continuous, but because they belong to the same functional surface, they still belong to the same semantic unit. For example, although the scattered sampling points on the outer panel of a car door are separated, they all belong to the same functional surface. At the same time, sampling points belonging to different functional surfaces, even if spatially adjacent, can be distinguished by their surface affiliation. For example, sampling points on the edge of a car door and adjacent pillars can be identified as belonging to different surfaces. Based on these characteristics, point cloud data can accurately depict the local geometric details of each surface, including curvature, normal, contour, thickness, and other local morphology. It can uniformly represent the geometric features of the same surface and effectively distinguish spatially adjacent but functionally different regions, avoiding misidentification and confusion of sampling points from different surfaces.

[0031] In step 202, each surface in the 3D model is pre-configured with a unique surface ID. This surface ID serves as an identifier to distinguish different surfaces, ensuring that each surface has a unique and non-repeating identity.

[0032] For point cloud data, the server iterates through each sampling point, matches the three-dimensional spatial coordinates of the sampling point with the coordinate range of each surface in the three-dimensional model, substitutes the three-dimensional coordinates of the sampling point into the coordinate range of each surface, determines whether it falls within the coordinate range of a certain surface, and then determines the surface to which the sampling point belongs. Finally, the surface ID corresponding to the surface is bound to the sampling point, so that each sampling point is matched one-to-one with the surface ID of the surface to which it belongs.

[0033] For multi-view 2D images, the server determines the foremost 2D image where each pixel is located in the multi-view image through depth testing. This foremost 2D image corresponds to a unique surface in the 3D model. Then, based on the surface ID of the surface corresponding to this foremost 2D image, the pixel is bound to the corresponding surface ID.

[0034] In step 203, the server invokes a preset semantic segmentation model, using the face ID as the association benchmark, to achieve cross-modal attention fusion of point cloud data and multi-view 2D images. Point cloud data can represent local geometric details of a surface, but lacks global spatial context, and sampling points on the same surface are easily scattered in space. Multi-view 2D images, on the other hand, provide global spatial association, positional relationships, and occlusion information, compensating for the shortcomings of point clouds in overall semantic understanding. This application's cross-modal fusion based on face ID consistency can force the semantic segmentation model to prioritize associating and aggregating all cross-modal information belonging to the same surface, integrating features of the same surface scattered across different views and point cloud locations into fused point cloud features.

[0035] This application uses surface ID as a hard constraint to ensure that the same surface is not incorrectly segmented or confused with other surfaces, strictly adhering to the original topological structure of the 3D model. Furthermore, through cross-modal attention fusion, it utilizes both the local geometric features of the point cloud to ensure no loss of surface details and introduces the global context of multi-view 2D images, enabling the semantic segmentation model to understand the function and position of the surface within the overall structure. This achieves a unification of microscopic geometric details and macroscopic spatial relationships, resulting in fused point cloud features. These fused point cloud features can simultaneously carry both geometric detail information and spatial context information from each surface, thus clearly distinguishing surfaces with different functions even in complex assembly, occlusion, overlapping, and spatial proximity scenarios.

[0036] The server calls a semantic segmentation model to segment the fused point cloud features, obtaining the functional region label of each sampling point in the fused point cloud features.

[0037] In step 204, the server iterates through each sampling point in the fused point cloud features one by one, maps the functional region label corresponding to the sampling point to the face ID bound to the sampling point, and finally each face ID uniquely corresponds to a functional region label, thus obtaining the correspondence between each face ID and the functional region label.

[0038] In step 205, the server categorizes and organizes all face IDs based on their functional area labels, grouping face IDs with the same functional area label into a single surface set. All surfaces corresponding to face IDs within the same surface set have identical functional attributes; therefore, each surface set can be considered a functional component.

[0039] The server can also assign a unique part identifier to each set of surfaces, which is used to clearly distinguish different functional parts. For example, the sampling points of the hood may correspond to a set of surfaces with surface IDs {101,102,205,307}. The server automatically assigns a unique part identifier to this set of surfaces and gives it a meaningful part name, such as HOOD, based on the functional semantics of the hood.

[0040] In this application, point cloud data and multi-view 2D images of a 3D model are acquired. Each sampling point in the point cloud data and each pixel in the multi-view 2D images are then bound to a face ID of the 3D model. Using the face ID as the association, cross-modal attention fusion is performed on the point cloud data and multi-view 2D images corresponding to the same face ID. Leveraging the strong constraint of the face ID, this ensures that the same surface is not incorrectly segmented or confused with other surfaces. Simultaneously, cross-modal fusion considers both the local geometric details of each surface in the point cloud and the global spatial structure of the multi-view 2D images, resulting in fused point cloud features. Based on these fused point cloud features, functional region labels for each sampling point are obtained. By mapping the functional region label of each sampling point to the corresponding face ID bound to that sampling point, a one-to-one correspondence between face IDs and functional region labels is established. Face IDs with the same functional region label are then merged into a surface set, ultimately determining each functional part of the 3D model. This application achieves accurate identification of each functional part of the 3D model through cross-modal fusion constrained by face IDs and label mapping.

[0041] As an optional implementation, in step 203, using a preset semantic segmentation model and with the face ID as the association reference, point cloud data and multi-view 2D images are fused across modal attention to obtain fused point cloud features. The functional region label of each sampling point in the fused point cloud features is then determined to include: Step S11: Input the point cloud data and multi-view 2D images into the preset semantic segmentation model; Step S12: The image encoder in the semantic segmentation model performs feature aggregation on the multi-view two-dimensional image to obtain image features, and the point cloud encoder in the semantic segmentation model extracts features from the point cloud data to obtain point cloud features. Among them, the image features can capture the feature correlation of the same surface under multiple views, and the point cloud features can distinguish regions that are physically close but belong to different parts. Step S13: Perform cross-modal attention fusion on image features and point cloud features through a cross-modal attention layer, and associate image features and point cloud features corresponding to the same face ID during the fusion process to obtain fused point cloud features. The image encoder and the point cloud encoder share the cross-modal attention layer. Step S14: Decode and classify the fused point cloud features using a decoder to obtain the functional region label for each sampling point.

[0042] In step S11, the server inputs the point cloud data with completed face ID binding and the multi-view 2D image into a preset semantic segmentation model. This semantic segmentation model adopts a structure of dual encoders, a cross-modal attention layer, and a single decoder. The image encoder and the point cloud encoder are parallel branches, processing the multi-view 2D image and point cloud data respectively. The outputs of both encoders are connected to the input of a shared cross-modal attention layer, and the output of the cross-modal attention layer is connected to the decoder. Finally, the decoder uniformly completes the final classification label output, forming an end-to-end inference structure from multi-modal input to functional region label output. In this application, the two parallel branches share a cross-modal attention layer, which is beneficial for learning a unified cross-modal representation and reducing the number of parameters. The encoder in this application can be PointNet++.

[0043] In step S12, the server performs layer-by-layer convolution and context feature aggregation on the multi-view 2D images through the image encoder in the semantic segmentation model to obtain the corresponding image features. Since multi-view 2D images can present the visible range, spatial position relationship, and occlusion state of surfaces under different viewpoints, the image encoder can fully learn the feature continuity of the same surface under multiple views, the relative positional relationship between surfaces, and the occlusion rules under different viewpoints during the convolution aggregation process. Therefore, the output image features can characterize the surface spatial association, relative positional relationship of surfaces, and viewpoint occlusion information of the 3D model.

[0044] Simultaneously, the server extracts 3D geometric features from the point cloud data using a point cloud encoder within the semantic segmentation model, obtaining the corresponding point cloud features. Point cloud data consists of sampling points carrying 3D coordinates and surface IDs, accurately reflecting the local morphology of a surface, such as curvature, normal, contour, and thickness. Therefore, the point cloud encoder can extract robust local geometric features based on the 3D spatial distribution and surface ID attribution, clearly distinguishing discontinuous regions within the same surface, as well as physically adjacent regions belonging to different surfaces, avoiding classification confusion caused by spatial proximity.

[0045] In step S13, the server performs cross-modal attention fusion on image features and point cloud features through a cross-modal attention layer shared by the image encoder and point cloud encoder in the semantic segmentation model. During the fusion calculation, the surface ID is used as the association benchmark to associate and aggregate image features and point cloud features belonging to the same surface ID. This fusion method uses the surface ID as a hard constraint to ensure that cross-modal features corresponding to the same surface are preferentially aggregated, preventing incorrect segmentation and confusion with features corresponding to other surfaces, while adhering to the original topological structure of the 3D model. During the fusion process, the microscopic geometric details carried by the point cloud features are preserved, while the macroscopic spatial structure of the image features is introduced, enabling the semantic segmentation model to simultaneously possess refined local geometric perception and global structural understanding capabilities, ultimately obtaining fused point cloud features that take into account both microscopic details and global semantics.

[0046] In step S14, the server decodes and classifies the fused point cloud features point by point using the decoder in the semantic segmentation model. The decoder determines the functional region to which each sampling point belongs based on the local geometric details, global spatial structure, and surface ID constraint information contained in the fused point cloud features, and finally outputs the functional region label corresponding to each sampling point in the point cloud data.

[0047] In this application, point cloud data bound to surface IDs and multi-view 2D images are input into a semantic segmentation model with a dual encoder structure. The image encoder extracts image features that can characterize the spatial association of surfaces, the relative positional relationship of surfaces, and viewpoint occlusion information. At the same time, the point cloud encoder extracts point cloud features to distinguish different neighboring functional regions. Then, with the help of the cross-modal attention layer shared by the dual encoders, the two types of features are fused with surface ID as the association benchmark. This can fully combine the microscopic geometric accuracy of point clouds and the global semantic advantages of multi-view images, while ensuring that the same surface is not incorrectly segmented and is not confused with other surfaces. Finally, the decoder accurately outputs the functional region label of each sampling point.

[0048] As an optional implementation, in step S12, the image features obtained by feature aggregation of the multi-view two-dimensional images through the image encoder in the semantic segmentation model include: mapping the face ID bound to each pixel in the multi-view two-dimensional image to a feature vector through the image encoder; performing neighborhood aggregation on the feature vector of each pixel in a single two-dimensional image in the spatial dimension to obtain the global feature vector of the single view corresponding to the two-dimensional image; fusing the global feature vectors of all two-dimensional images and associating the global feature vectors of the same face ID under the multi-view to obtain the image features.

[0049] The server uses a face ID embedding layer within the image encoder to map each integer-identified face ID bound to each pixel in a multi-view 2D image into a dense, high-dimensional, and learnable feature vector. This transforms discrete face IDs into continuous feature representations suitable for deep learning networks. Then, a lightweight convolutional network, such as several layers of a CNN (Convolutional Neural Network), aggregates the feature vectors of all pixels belonging to the same face ID within a single 2D image, using the face ID as the grouping criterion. This is done by averaging or weighted summation, eliminating the spatial dispersion of pixel features within the same face ID. This yields a single-view feature vector for that face ID from the current perspective, ensuring that the feature representation of the same surface remains consistent within a single image.

[0050] The image encoder traverses all multi-view 2D images and performs cross-view fusion on the single-view feature vectors generated for the same face ID under different effective viewpoints. In this embodiment, the fusion is performed by averaging the values ​​to remove invalid information caused by viewpoint occlusion and unify the feature representation of the face ID under multiple viewpoints. For example, there are 3 2D images from different angles, all of which capture the 5th face of the same 3D object. The first view image captures the 5th face, and its feature vector is V1. The second view image captures the 5th face, and its feature vector is V2. The 5th face in the third view image is occluded and is not captured. Therefore, the feature vector of the 5th face is the average of V1 and V2.

[0051] Finally, the image encoder concatenates the feature vectors obtained by fusing all face IDs from multiple perspectives in a fixed order to form a complete image feature. This image feature uses face IDs as the basic unit, combining spatial consistency within a single perspective with global correlation between multiple perspectives, and can accurately characterize the global spatial structure, relative positional relationships, and viewpoint occlusion information of each surface of the 3D model.

[0052] As an optional implementation, in step S13, cross-modal attention fusion is performed on image features and point cloud features through a cross-modal attention layer, and image features and point cloud features corresponding to the same face ID are associated during the fusion process to obtain fused point cloud features including: Step S131: Unify the feature channels of image features and point cloud features through a cross-modal attention layer; Step S132: Based on surface ID consistency, attention-weighted fusion is performed on the unified image features and point cloud features to obtain the first fused point cloud features, wherein the first fused point cloud features are used to indicate the preliminary association between the image features and the point cloud features; Step S133: Add the residuals of the first fused point cloud features and the unified channel point cloud features and normalize them to obtain the second fused point cloud features. The second fused point cloud features are used to incorporate the correlation between image features and point cloud features while retaining the point cloud features. Step S134: After the second fused point cloud feature is mapped by the feedforward network feature, the residuals of the second fused point cloud feature are added and normalized to obtain the final fused point cloud feature.

[0053] In step S131, the server processes image features through a fully connected linear transformation layer inside the cross-modal attention layer. ∈ Point cloud features ∈ Channel-dimensional mapping is performed separately to unify the two into the same L-dimensional feature space, resulting in channel-aligned image features. and point cloud features ,in ∈ , ∈ This is a learnable weight matrix. This operation eliminates the feature dimension mismatch problem between the image branch and the point cloud branch caused by differences in network structure, ensuring that cross-modal features can be computed in the same vector space. In the formulas for the image features and point cloud features mentioned above, R is in the real number field, indicating that the elements in the tensor are real numbers; H and W represent the height and width of the image feature map, respectively; C represents the number of channels in the image feature map; N represents the number of sampling points; and D represents the feature dimension, i.e., the length of the feature vector at each sampling point.

[0054] In step S132, the server performs attention-weighted fusion of the unified image features and point cloud features based on surface ID consistency to obtain the first fused point cloud feature, which is used to indicate the preliminary association between the image features and the point cloud features.

[0055] The process of obtaining the first fused point cloud features through attention-weighted fusion includes: generating a query vector based on the point cloud features after unifying the channels, and generating a key vector and a value vector based on the image features after unifying the channels. The query vector indicates the features of each sampling point, the key vector indicates the features of each pixel, and the value vector indicates the feature value of each pixel. Iterating through each sampling point and each pixel, a bias term is determined based on whether the face IDs bound to the sampling point and the pixel are consistent. The bias term corresponding to consistent face IDs is greater than the bias term corresponding to inconsistent face IDs. Attention-weighted calculation is performed based on the query vector, key vector, value vector, and bias term to obtain the first fused point cloud features.

[0056] The server utilizes dimension-aligned point cloud features. Generate query vectors Utilizing dimension-aligned image features Generate key vectors AND value vector ,in, , , This is a learnable weight matrix used to map input features to a unified attention computation dimension.

[0057] The query vector is an N× The matrix is ​​a set of points, where each element represents a feature query for a point cloud sampling point, used to match corresponding semantic information in image features. N is the number of point cloud sampling points. Dimensions are calculated for attention.

[0058] The key vector is an H×W× The matrix is ​​a set of features, where each element represents a feature key at a location in the image feature map, used to calculate similarity with the query vector. H and W are the height and width of the image feature map, respectively. Dimensions are calculated for attention.

[0059] The value vector is an H×W× A matrix, where each element represents a feature value at a location in the image feature map, is used to aggregate supplementary information into point cloud features based on similarity weights. To output feature dimensions, usually with Consistent.

[0060] When calculating attention weights, this application introduces a structured bias term B∈ based on surface ID matching degree. If the i-th sampling point in the point cloud corresponds to the same face ID as the j-th pixel in the image feature map, then = ,otherwise =0, This represents the bias term between the i-th sampling point and the j-th pixel.

[0061] =0 indicates that the sampling points and pixels belong to different surfaces of the 3D model. The sampling points and pixels have no topological relationship. No additional bias guidance is applied in the attention weighting calculation. The weight of feature matching is determined only by the similarity between the point cloud and the image's own features.

[0062] = This indicates that the sampled points and pixels belong to the same surface of the 3D model and have a clear topological relationship. In the attention weighting calculation, a targeted bias will be applied to guide the semantic segmentation model to prioritize and aggregate cross-modal features of the same surface, and suppress erroneous matching of features between different surfaces.

[0063] also, The magnitude of directly reflects the bias guidance strength for cross-modal feature pairs of the same-face ID. The larger the value, the stronger the bias of the sampling point-pixel pairs of the same surface, which can further increase its weight in attention calculation, allowing the semantic segmentation model to prioritize the association and fusion of the point cloud features of this same surface with the image features, and strengthen the constraint of the surface ID topological prior. The smaller the value, the weaker the bias of the sampling point-pixel pairs of the same surface. The semantic segmentation model has a milder tendency to prioritize the fusion of the features of the same surface, and it is more inclined to the latter between topological constraints and the semantic matching of the features themselves.

[0064] Therefore, the bias term Its function is to introduce the surface ID topological prior information of the 3D model as a hard constraint for cross-modal attention fusion, through 0 and The numerical values ​​distinguish between different surfaces and cross-modal feature pairs on the same surface, and then through... The size adjustment affects the guiding strength of feature pairs on the same surface, thereby explicitly guiding the semantic segmentation model to prioritize the aggregation of point cloud features and image features corresponding to the same surface during cross-modal feature matching.

[0065] Subsequently, the cross-modal attention layer calculates the dot product of the query vector and the key vector, adds a bias term, normalizes it using Softmax, and then combines it with the key vector to obtain the attention weights. The formula for calculating the attention weights is as follows:

[0066] in, This represents the attention weight, i.e., the first fused point cloud feature, and Q is the query vector. Let L be the transpose of the key vector, L be the unified channel, γ be a hyperparameter used to adjust the constraint strength of the surface ID prior, B be the bias term, and V be the value vector.

[0067] γ is a fixed value between 0.5 and 1, used to amplify or reduce the bias term B through weighting. The larger γ is, the stronger the constraint of the face ID prior, and the more significant the biasing effect on the point cloud sampling points and image pixel pairs with the same face ID. This allows the semantic segmentation model to rigidly prioritize the association of cross-modal features corresponding to the same face ID, almost unaffected by the semantic similarity between the point cloud and the image itself, while strongly suppressing erroneous matching between dissimilar ID feature pairs. The smaller γ is, the weaker the constraint of the face ID prior, and the more moderate the biasing effect on the same face ID feature pairs. The semantic segmentation model exhibits a flexible guiding characteristic in its priority fusion of same face ID features. In this case, the semantic segmentation model will refer more to the semantic correlation between the point cloud and the image itself for cross-modal fusion based on the face ID topological prior. If the dissimilar ID feature pairs have high semantic similarity, their attention weights will be closer to the same face ID feature pairs. The rigidity of the constraint is reduced, but the fusion priority of same face ID features is still retained.

[0068] In the above formula, attention weights are obtained by associating cross-modal features from the same surface to suppress erroneous matching caused by viewpoint occlusion and appearance similarity. These attention weights are the first fused point cloud features. A preliminary accurate correspondence between image features and point cloud features, guided by topological identity, was established.

[0069] In step S133, the server will merge the first point cloud features Original point cloud features after channel unification Perform residual summation and then perform layer normalization. Processing yields the second fused point cloud features. The formula for calculating the second fused point cloud features is:

[0070] In the above formula, the residual structure can completely preserve the accurate three-dimensional geometric details in the original point cloud features, while the normalization operation stabilizes the feature distribution and alleviates the gradient vanishing problem. This allows the second fused point cloud features to effectively integrate the cross-modal correlation information between image features and point cloud features without losing the local geometric accuracy of the point cloud, taking into account both microscopic geometric details and global spatial semantics.

[0071] In step S134, the server will merge the second point cloud features. The input is processed by a feedforward network (FFN) for nonlinear feature mapping and high-order semantic extraction, and then the output of the feedforward network is processed. and Perform a second residual summation and then perform layer normalization again. The process yields the final fused point cloud features. The final formula for calculating the fused point cloud features is:

[0072] This process further enhances feature representation capabilities, uncovers higher-order dependencies between cross-modal features, and enables the fused point cloud features to simultaneously possess the high-precision geometric information of the point cloud, the global contextual information of the image, and the topological consistency brought about by the surface ID constraint.

[0073] This application employs a feature-level fusion strategy for multimodal fusion, completing the feature interaction between the image and point cloud in the intermediate layer of the semantic segmentation model. It does not require precise point-to-point alignment between image pixels and point cloud sampling points in physical space. Instead, it establishes cross-modal semantic associations in a high-dimensional feature space through an attention mechanism guided by surface ID. For example, the local geometric details of the same surface belonging to the hood in the point cloud can be automatically associated with all regions presenting that surface in multi-view images using the attention mechanism constrained by surface ID consistency. Even if the surface is obscured by perspective, distorted by perspective, or partially visible in different views, the semantic segmentation model can still accurately identify and aggregate cross-modal features.

[0074] This fusion method, compared to early fusion of pixels and sampling points, avoids the pixel-point cloud spatial alignment problem caused by viewpoint distortion, occlusion, and differences in sampling density. It achieves cross-modal interaction without complex coordinate registration, reducing the dependence on data preprocessing accuracy. Furthermore, compared to late-stage fusion at the decision level, completing information interaction at the feature level allows for earlier complementarity between the global semantics of the image and the geometric details of the point cloud. This avoids information fragmentation and insufficient interaction problems caused by merging after independent decisions from each modality, improving the richness and robustness of feature representation. In other words, this application establishes cross-modal semantic associations in the feature space through an attention mechanism guided by surface ID. This avoids the spatial alignment problem of early fusion and compensates for the insufficient interaction problem of late fusion. While preserving the integrity of information from each modality, it achieves efficient and robust cross-modal feature aggregation, improving the semantic recognition and segmentation accuracy of the semantic segmentation model for 3D surfaces.

[0075] This application employs a unified feature channel, attention-weighted fusion guided by surface ID, two-layer residual connections and layer normalization, and high-order mapping processing via feedforward networks. While strictly preserving the original local geometric accuracy of the point cloud, it fully incorporates the global spatial structure, positional relationships, and occlusion information from multi-view images. Simultaneously, by leveraging a surface ID consistency bias term, it explicitly guides the model to focus on cross-modal features belonging to the same surface ID, suppressing incorrect matching and ensuring that the same surface is not incorrectly segmented and different surfaces are not confused. Ultimately, this yields high-quality fused point cloud features that combine microscopic geometric details, global spatial correlation, and engineering topological consistency.

[0076] As an optional implementation, in step S14, the decoder performs decoding and classification on the fused point cloud features to obtain the functional region label for each sampling point. This includes: mapping the features of each sampling point in the fused point cloud features to the dimensions corresponding to each functional region label using the decoder to obtain the score of each sampling point under each functional region label; normalizing the scores to obtain the prediction probability of each sampling point under each functional region label dimension; and selecting the functional region label corresponding to the highest prediction probability as the functional region label of the sampling point.

[0077] The server uses a fully connected layer in the decoder to linearly map the high-dimensional feature vector of each sample point in the fused point cloud features to an output dimension consistent with the number of functional region labels. This yields a score for each sample point under each functional region label, reflecting the degree of matching between the sample point's features and the semantic templates of different functional regions. The decoder then performs Softmax normalization on the score of each sample point, converting it into a predicted probability distribution with values ​​ranging from [0,1] and summing to 1. This allows the semantic segmentation model to output predicted probabilities that allow direct sampling points to belong to their respective functional regions. Finally, the functional region label corresponding to the maximum value in the predicted probability distribution is selected as the final functional region label for that sample point, achieving accurate identification of the functional region to which each sample point in the 3D model belongs.

[0078] This application utilizes the decoder's dimension mapping, probability normalization, and maximum probability label selection to efficiently convert the local geometric details, global spatial structure, and surface ID topological constraint information contained in the fused point cloud features into functional region labels for each sampling point. This ensures that the classification results possess both point cloud geometric accuracy and conform to the topological structure and engineering semantics of the 3D model.

[0079] Figure 3 The flowchart of the semantic segmentation model shows that the processing flow of the semantic segmentation network is divided into four stages: input layer, dual-branch encoder, cross-modal attention layer, and output layer.

[0080] 1. The input layer receives two types of input data.

[0081] Multi-view 2D images: generated by projecting and rendering the 3D model along the principal axis and six views, used to provide information on the surface space association, relative positional relationship of surfaces, and view occlusion of the 3D model.

[0082] Point cloud data: obtained by discrete sampling of the surface of a 3D model, carrying 3D coordinates and surface ID binding information, used to provide local geometric details of the surface.

[0083] 2. Dual-branch encoder layer.

[0084] A parallel dual encoder structure is adopted to extract features independently from the two types of modal data.

[0085] Image encoder: Performs global context aggregation on multi-view 2D images and outputs image features. It is used to characterize the feature correlation of the same surface in a 3D model under multiple viewpoints.

[0086] Point cloud encoder: Extracts local geometric features from point cloud data and outputs point cloud features. It is used to accurately depict the curvature, normal and other details of curved surfaces, and to distinguish areas that are physically adjacent but belong to different parts.

[0087] 3. Shared cross-modal attention layer.

[0088] As the core of the network, image features and point cloud features are input together into a cross-modal attention layer with shared parameters. Using face IDs as the association criterion, cross-modal features with the same face ID are weighted and aggregated in the feature space to suppress erroneous matching of features with different face IDs. The fused point cloud features are then output. This feature preserves both the local geometric details of the point cloud and the global spatial structure of the image, while strictly adhering to the topological structure of the 3D model.

[0089] 4. Output layer.

[0090] The fused point cloud feature input decoder: The decoder decodes and classifies the fused features point by point, maps the high-dimensional features to the probability distribution of functional region labels, and finally outputs the semantic segmentation result, that is, the functional region label corresponding to each sampling point in the point cloud.

[0091] As an optional implementation method, the training process of the semantic segmentation model includes: Acquire training data and input the training data into the initial semantic segmentation model. The training data includes sample multi-view two-dimensional images, sample point cloud data, and the actual functional region label corresponding to each sampling point. The initial semantic segmentation model is used to process the multi-view two-dimensional images and point cloud data of the samples, and the predicted functional region labels of each sampling point are output. The loss between the predicted functional region label and the actual functional region label is calculated based on the joint loss function, which is constructed based on weighted cross-entropy loss, Dessian loss and in-plane consistency loss. During backpropagation, the parameters of the image encoder, point cloud encoder, cross-modal attention layer, and decoder are updated together based on the loss signals of the image encoder and point cloud encoder until the model converges, resulting in a trained semantic segmentation model.

[0092] The semantic segmentation model training process employs an end-to-end joint training strategy, with the following steps: The server acquires the training dataset, including multi-view 2D images of samples, sample point cloud data, and the actual functional region label corresponding to each sampling point, i.e., the ground truth label. To accelerate model convergence, the image encoder and point cloud encoder are initialized with weights pre-trained on large general datasets, such as ImageNet and ShapeNet.

[0093] In the forward propagation and prediction output stage, the server simultaneously feeds the paired sample multi-view 2D images and sample point cloud data into the initial semantic segmentation model to perform forward inference. The image encoder and point cloud encoder extract their respective modal features. The two types of features are input into a cross-modal attention layer with shared parameters. This layer is called by both branches to complete cross-modal feature aggregation based on surface ID consistency. The decoder decodes the fused point cloud features point by point and outputs the predicted functional region label for each sampling point.

[0094] During the joint loss calculation phase, the server calculates the error between the predicted functional region labels and the actual functional region labels based on the joint loss function. The joint loss function consists of three weighted loss types: weighted cross-entropy loss, Descein loss, and intra-plane consistency loss. The weighted cross-entropy loss alleviates the problem of imbalanced sample classes and improves the ability to identify rare functional regions; the Descein loss optimizes region overlap, improving the continuity and accuracy of segmentation boundaries; and the intra-plane consistency loss utilizes the unique face ID supervision signal of the 3D model data to ensure that the predicted labels of all sampling points under the same face ID remain consistent, strengthening topological compliance. This joint loss function fully utilizes the engineering topology information of the 3D model, making the training objective more closely aligned with actual engineering needs.

[0095] The formulas for each loss function are shown below.

[0096] ,in, For the joint loss function, For the loss of Des, For weighted cross-entropy loss, For in-plane consistency loss, , These are the weighting coefficients, where, , The sum of is 1. .

[0097] ,in, The weighted cross-entropy loss is used, where C is the total number of label categories. The weights assigned to label category c, Typically, the weight is inversely proportional to the class frequency; that is, the fewer the number of classes, the higher the weight. The larger, For the actual functional area label, if the pixel belongs to category c, then =1, otherwise 0 Represented in one-hot encoding form, The probability of a pixel belonging to label category c is predicted by the output of the Softmax function, with a value between 0 and 1.

[0098] ,in, The loss is called Dessian loss, where X represents the set of pixels predicted to belong to a certain label category, and Y represents the set of pixels in the true label that belong to that label category.

[0099] ,in, The loss function is called the in-plane consistency loss, where K is the total number of distinct face IDs in a training sample. Let be the set of all sampled points of the k-th face ID. For set The number of sampling points in the middle Let be the class probability vector predicted by the semantic segmentation model for the i-th sampling point. For set The mean vector of the predicted probability vectors of all sampled points. It is the squared L2 norm of the vector, used to measure the difference between the class probability vector of a single sample point and the mean vector of the group.

[0100] During the backpropagation and parameter update backpropagation stages, the gradient of the cross-modal attention layer is jointly contributed by the loss signals of the image encoder and the point cloud encoder, realizing a dual-branch joint update. The optimizer uses Adam (Adaptive Moment Estimation) or SGD (Stochastic Gradient Descent), and the learning rate adopts a multinomial decay or warm-up strategy to balance training stability and convergence speed. All parameters of the image encoder, point cloud encoder, shared cross-modal attention layer and decoder are updated simultaneously, so that the attention layer learns a fusion strategy that is beneficial to both modalities.

[0101] During the data augmentation and convergence determination phase, data augmentation strategies are introduced during training, including random scaling, rotation, and random horizontal flipping, to perturb the input image and point cloud data, expand data diversity, and improve the model's generalization ability. The forward propagation, loss calculation, and backpropagation processes are repeated until the model's loss on the validation set no longer decreases and the performance metrics tend to stabilize, resulting in a trained semantic segmentation model.

[0102] This application utilizes pre-trained weight initialization, end-to-end joint training, multi-objective joint loss, and data augmentation to accelerate convergence by leveraging prior knowledge from general datasets and fully exploiting the unique face ID supervision signals of 3D model data. This enables the semantic segmentation model to maintain the topological consistency of the 3D model while ensuring the accuracy of functional region recognition. The joint update mechanism with shared attention layers makes the cross-modal fusion strategy more adaptable to bimodal features, ultimately improving the robustness and engineering applicability of the model in complex 3D model scenarios.

[0103] Figure 4 This is a schematic diagram of the overall process of this application, including the following content.

[0104] 401. 3D Model Input: Receives the 3D model file to be processed.

[0105] 402. Data Representation and Extraction: Convert the original 3D model into a multimodal data format adapted for deep learning; perform multi-view projection rendering on the 3D model to generate multi-view 2D images to characterize the global spatial structure of the 3D model; perform discrete sampling on the surface of the 3D model to generate point cloud data with face ID binding to accurately depict the geometric details of the surface; complete the face ID binding between point cloud sampling points and image pixels.

[0106] 403. Semantic Segmentation Model Processing: A semantic segmentation network with dual encoders, a shared cross-modal attention layer, and a decoder is used to perform end-to-end inference on the input multi-view 2D images and point cloud data: the image encoder and the point cloud encoder extract image features and point cloud features respectively; with surface ID as a hard constraint, feature-level fusion is completed through the shared cross-modal attention layer, taking into account both local geometric details and global spatial structure; the decoder classifies point by point and outputs the functional region label corresponding to each sampling point.

[0107] 404. Region Merging and Post-processing: Post-process the discrete sampling point labels output by the semantic segmentation model: aggregate sampling points with the same functional region label into a set of connected surfaces; map the functional region label of the sampling point to its bound surface ID, establish a one-to-one correspondence between surface ID and functional region label, and ensure the consistency of labels for the same surface.

[0108] 405. Grouping and Mesh Parameter Mapping: Based on the mapping relationship between face IDs and functional area labels, face IDs with the same functional area labels are grouped into independent functional parts. Each functional part is assigned a unique part identifier and standardized naming is completed. At the same time, each functional part is associated with the corresponding mesh generation parameters to generate a script file that can be directly imported into CAE software, realizing automatic grouping of functional parts and one-click mapping of mesh parameters.

[0109] 406. Script Generation: Encapsulates functional part grouping information, standardized naming rules, and differentiated meshing parameters into an executable script for ANSA (Automated Numerical Simulation Applications).

[0110] 407. Differentiated Mesh Generation: The script can be directly imported into CAE software, such as ANSA, to automatically complete part grouping, naming and mesh generation, realizing full automation of the CAD to CAE preprocessing process.

[0111] This application can achieve the following beneficial effects.

[0112] 1. A significant increase in efficiency.

[0113] This application replaces traditional manual grouping with a fully automated process, reducing the time for functional area division in CAE preprocessing to minutes. Engineers no longer need to manually select and classify thousands of 3D model surfaces; they only need to import the original 3D model, and the system can automatically complete multimodal data generation, semantic segmentation, surface ID mapping, and part grouping, all without human intervention. This eliminates the subjectivity and inconsistency of manual operation, ensures stable and reproducible grouping results, and significantly reduces the labor and time costs of CAE preprocessing.

[0114] 2. Improved grouping accuracy and engineering rationality.

[0115] This application utilizes deep learning-based functional semantic recognition to replace traditional grouping methods that rely on experience or simple rules, resulting in grouping results that better align with actual engineering needs. The semantic segmentation model can learn complex functional region patterns, such as stiffeners, thin walls, fillets, and load-bearing surfaces, directly identifying actual functional regions that conform to simulation analysis logic, rather than simply grouping based on part names or geometric shapes. Simultaneously, surface ID constraints ensure label consistency for the same surface, ultimately outputting a connected set of surfaces, avoiding fragmented and discrete problems, and better meeting the requirements of unified mesh control for the same surface in CAE analysis, preventing inconsistent mesh parameter application.

[0116] 3. Technological innovation and robustness improvement for engineering scenarios.

[0117] This application innovatively uses the inherent face IDs of 3D models as the core link for cross-modal fusion and topology backtracking, improving the segmentation robustness in complex scenes. By binding face IDs to point cloud sampling points and image pixels, a unified and traceable multimodal input is constructed. In the feature fusion stage, the face ID matching relationship is used as a bias term to guide the semantic segmentation model to prioritize cross-modal features from the same surface. This approach utilizes the global context of the image to perceive viewpoint occlusion and overall layout, while preserving the precise geometric details of the point cloud. At the same time, it strictly follows the topological logic of the 3D model, effectively solving the problem of incorrect matching in scenarios such as complex assemblies and viewpoint occlusion, making the segmentation results more accurate and stable.

[0118] 4. Supports differentiated grid partitioning to improve the quality of CAE analysis.

[0119] Based on the engineering attributes of functional areas, such as stress concentration, structural complexity, and simulation accuracy requirements, differentiated meshing strategies can be applied to different functional areas. For example, fine-grained high-density meshes can be used for rounded corners and connecting surfaces, while coarse-grained low-density meshes can be used for gently sloping structural areas, balancing computational accuracy and efficiency. Simultaneously, the automated grouping process ensures that the functional area division and mesh parameter configuration are completely consistent when the same 3D model is processed by different projects or different engineers, avoiding deviations caused by manual operation and improving the comparability and reliability of CAE analysis results.

[0120] Based on the same technical concept, this application also provides a schematic diagram of a functional part division device for a three-dimensional model, such as... Figure 5 As shown, it includes the following: The acquisition module 501 is used to acquire point cloud data and multi-view two-dimensional images of the three-dimensional model. The point cloud data is used to indicate the local geometric details of each surface of the three-dimensional model, and the multi-view two-dimensional images are used to indicate the global spatial structure of the three-dimensional model. The binding module 502 is used to bind the surface ID of the surface to each sampling point of the point cloud data based on the surface ID of each surface in the 3D model, and to bind the surface ID of the surface corresponding to the 2D plane to each pixel of the multi-view 2D image. The fusion module 503 is used to perform cross-modal attention fusion of point cloud data and multi-view two-dimensional images using a preset semantic segmentation model and with the surface ID as the association reference, to obtain fused point cloud features and determine the functional region label of each sampling point in the fused point cloud features. The mapping module 504 is used to map the functional area label of each sampling point to the face ID bound to the corresponding sampling point, so as to obtain the functional area label corresponding to each face ID. The allocation module 505 is used to merge face IDs with the same functional area label into a surface set and assign part identifiers to the surface set, wherein each surface set is used to indicate a functional part.

[0121] Optionally, the fusion module 503 is used for: Input point cloud data and multi-view 2D images into a preset semantic segmentation model; The image encoder in the semantic segmentation model aggregates features from multi-view 2D images to obtain image features, and the point cloud encoder in the semantic segmentation model extracts features from point cloud data to obtain point cloud features. The image features can capture the feature correlation of the same surface under multiple views, and the point cloud features can distinguish regions that are physically close but belong to different parts. Cross-modal attention fusion of image features and point cloud features is performed through a cross-modal attention layer. During the fusion process, image features and point cloud features corresponding to the same face ID are associated to obtain fused point cloud features. The image encoder and the point cloud encoder share the cross-modal attention layer. The fused point cloud features are decoded and classified using a decoder to obtain the functional region label for each sampling point.

[0122] Optionally, the fusion module 503 is specifically used for: The image encoder maps the face ID bound to each pixel in the multi-view 2D image to a feature vector; The feature vectors of all pixels corresponding to the same face ID in a single two-dimensional image are aggregated in the spatial dimension to obtain the single-view feature vector of each face ID in the single two-dimensional image. The single-view feature vectors of all two-dimensional images for the same face ID are fused to obtain the fused feature vector for each face ID; The fused feature vectors of each face ID are concatenated to obtain the image features.

[0123] Optionally, the fusion module 503 is specifically used for: Feature channels of image features and point cloud features are unified through a cross-modal attention layer; Based on surface ID consistency, attention-weighted fusion is performed on the unified image features and point cloud features to obtain the first fused point cloud feature, which is used to indicate the initial association between the image features and the point cloud features. The residuals of the first fused point cloud feature and the point cloud feature after unification are added and normalized to obtain the second fused point cloud feature. The second fused point cloud feature is used to incorporate the correlation between image features and point cloud features while retaining the point cloud features. The second fused point cloud feature is mapped through a feedforward network, and then the residuals of the second fused point cloud feature are added together and normalized to obtain the final fused point cloud feature.

[0124] Optionally, the fusion module 503 is specifically used for: A query vector is generated based on the point cloud features after unification, and a key vector and a value vector are generated based on the image features after unification. The query vector is used to indicate the features of each sampling point, the key vector is used to indicate the features of each pixel, and the value vector is used to indicate the feature value of each pixel. Iterate through each sampling point and each pixel, and determine the bias term based on whether the face IDs bound to the sampling point and the pixel are consistent. The bias term corresponding to consistent face IDs is greater than the bias term corresponding to inconsistent face IDs. Attention-weighted calculations are performed based on the query vector, key vector, value vector, and bias term to obtain the first fused point cloud features.

[0125] Optionally, the fusion module 503 is specifically used for: The decoder maps the features of each sampling point in the fused point cloud features to the dimension corresponding to each functional region label, thus obtaining the score of each sampling point under each functional region label. The scores are normalized to obtain the predicted probability of each sampling point under the corresponding dimension of each functional area label; Select the functional region label corresponding to the highest predicted probability as the functional region label of the sampling point.

[0126] Optionally, the device is also used for: Acquire training data and input the training data into the initial semantic segmentation model. The training data includes sample multi-view two-dimensional images, sample point cloud data, and the actual functional region label corresponding to each sampling point. The initial semantic segmentation model is used to process the multi-view two-dimensional images and point cloud data of the samples, and the predicted functional region labels of each sampling point are output. The loss between the predicted functional region label and the actual functional region label is calculated based on the joint loss function, which is constructed based on weighted cross-entropy loss, Dessian loss and in-plane consistency loss. During backpropagation, the parameters of the image encoder, point cloud encoder, cross-modal attention layer, and decoder are updated together based on the loss signals of the image encoder and point cloud encoder until the model converges, resulting in a trained semantic segmentation model.

[0127] like Figure 6As shown, this application provides an electronic device including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0128] Memory 603 is used to store computer programs.

[0129] In one embodiment of this application, when the processor 601 executes the program stored in the memory 603, it implements the functional part division method of the three-dimensional model provided in any of the foregoing method embodiments.

[0130] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the functional part division method for a three-dimensional model as provided in any of the foregoing method embodiments.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0133] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0134] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for dividing a three-dimensional model into functional parts, characterized in that, The method includes: The point cloud data and multi-view two-dimensional images of a three-dimensional model are acquired, wherein the point cloud data is used to indicate the local geometric details of each surface of the three-dimensional model, and the multi-view two-dimensional images are used to indicate the global spatial structure of the three-dimensional model. Based on the surface ID of each surface in the 3D model, each sampling point of the point cloud data is bound to the surface ID of the surface to which it belongs, and each pixel of the multi-view 2D image is bound to the surface ID of the surface corresponding to the 2D plane in which it is located. Using a pre-defined semantic segmentation model and with face ID as the association benchmark, the point cloud data and the multi-view 2D image are fused across modal attention to obtain fused point cloud features, and the functional region label of each sampling point in the fused point cloud features is determined. Map the functional area label of each sampling point to the face ID bound to the corresponding sampling point to obtain the functional area label corresponding to each face ID; Face IDs with the same functional area label are merged into a surface set, where each surface set is used to indicate a functional part.

2. The method according to claim 1, characterized in that, Using a pre-defined semantic segmentation model and with face ID as the association criterion, the point cloud data and the multi-view 2D image are fused across modal attention to obtain fused point cloud features. The functional region label of each sampling point in the fused point cloud features is determined to include: The point cloud data and the multi-view two-dimensional image are input into a preset semantic segmentation model; The image encoder in the semantic segmentation model performs feature aggregation on the multi-view two-dimensional image to obtain image features, and the point cloud encoder in the semantic segmentation model extracts features from the point cloud data to obtain point cloud features. The image features can capture the feature correlation of the same surface under multiple views, and the point cloud features can distinguish regions that are physically close but belong to different parts. The image features and the point cloud features are fused by a cross-modal attention layer, and the image features and point cloud features corresponding to the same face ID are associated during the fusion process to obtain fused point cloud features. The image encoder and the point cloud encoder share the cross-modal attention layer. The fused point cloud features are decoded and classified using a decoder to obtain the functional region label for each sampling point.

3. The method according to claim 2, characterized in that, The image features obtained by performing feature aggregation on the multi-view two-dimensional image through the image encoder in the semantic segmentation model include: The image encoder maps the face ID bound to each pixel in the multi-view 2D image to a feature vector. The feature vectors of all pixels corresponding to the same face ID in a single two-dimensional image are aggregated in the spatial dimension to obtain the single-view feature vector of each face ID in the single two-dimensional image. The single-view feature vectors of all two-dimensional images for the same face ID are fused to obtain the fused feature vector for each face ID; The fused feature vectors of each face ID are concatenated to obtain the image features.

4. The method according to claim 2, characterized in that, The image features and the point cloud features are fused using a cross-modal attention layer. During the fusion process, image features and point cloud features corresponding to the same face ID are associated to obtain fused point cloud features, including: Feature channels of the image features and the point cloud features are unified through a cross-modal attention layer; Based on surface ID consistency, attention-weighted fusion is performed on the unified image features and point cloud features to obtain a first fused point cloud feature, wherein the first fused point cloud feature is used to indicate the preliminary association between the image features and the point cloud features. The residuals of the first fused point cloud feature and the point cloud feature after unifying the channel are added and normalized to obtain the second fused point cloud feature. The second fused point cloud feature is used to incorporate the image features and the correlation between the point cloud features while retaining the point cloud features. After the second fused point cloud feature is mapped by the feedforward network feature, the residuals of the second fused point cloud feature are added together and normalized to obtain the final fused point cloud feature.

5. The method according to claim 4, characterized in that, Based on surface ID consistency, attention-weighted fusion is performed on the channel-unified image features and point cloud features to obtain the first fused point cloud features, which include: A query vector is generated based on the point cloud features after unification, and a key vector and a value vector are generated based on the image features after unification. The query vector is used to indicate the features of each sampling point, the key vector is used to indicate the features of each pixel, and the value vector is used to indicate the feature value of each pixel. Traverse each sampling point and each pixel, and determine the bias term based on whether the face IDs bound to the sampling point and the pixel are consistent. The bias term corresponding to consistent face IDs is greater than the bias term corresponding to inconsistent face IDs. Attention weighting is performed based on the query vector, key vector, value vector and bias term to obtain the first fused point cloud feature.

6. The method according to claim 2, characterized in that, The fused point cloud features are decoded and classified using a decoder to obtain the functional region label for each sampling point, including: The decoder maps the features of each sampling point in the fused point cloud features to the dimension corresponding to each functional region label, thereby obtaining the score of each sampling point under each functional region label. The scores are normalized to obtain the predicted probability of each sampling point in the corresponding dimension of each functional area label; The functional region label corresponding to the highest predicted probability is selected as the functional region label of the sampling point.

7. The method according to claim 1, characterized in that, The training process of the semantic segmentation model includes: Acquire training data and input the training data into the initial semantic segmentation model, wherein the training data includes sample multi-view two-dimensional images, sample point cloud data, and actual functional region labels corresponding to each sampling point; The initial semantic segmentation model is used to process the multi-view two-dimensional images of the samples and the point cloud data of the samples, and output the predicted functional region labels of each sampling point. The loss between the predicted functional region label and the actual functional region label is calculated based on the joint loss function, wherein the joint loss function is constructed based on weighted cross-entropy loss, Dessian loss and in-plane consistency loss; During backpropagation, the parameters of the image encoder, the point cloud encoder, the cross-modal attention layer, and the decoder are updated together based on the loss signals of the image encoder and the point cloud encoder until the model converges, resulting in a trained semantic segmentation model.

8. A functional parts division device for a three-dimensional model, characterized in that, The device includes: The acquisition module is used to acquire point cloud data and multi-view two-dimensional images of a three-dimensional model. The point cloud data is used to indicate the local geometric details of each surface of the three-dimensional model, and the multi-view two-dimensional images are used to indicate the global spatial structure of the three-dimensional model. The binding module is used to bind the surface ID of the surface to each sampling point of the point cloud data based on the surface ID of each surface in the 3D model, and to bind the surface ID of the surface corresponding to the 2D plane to each pixel of the multi-view 2D image. The fusion module is used to perform cross-modal attention fusion of the point cloud data and the multi-view two-dimensional image using a preset semantic segmentation model and with the face ID as the association reference, to obtain fused point cloud features and determine the functional region label of each sampling point in the fused point cloud features. The mapping module is used to map the functional area label of each sampling point to the face ID bound to the corresponding sampling point, so as to obtain the functional area label corresponding to each face ID; The assignment module is used to merge face IDs with the same functional area label into a surface set, where each surface set is used to indicate a functional part.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images