Annotation method, apparatus, computing device cluster, and computer program product

Abstract

The application provides a labeling method and device, a computing device cluster and a computer program product, and applies to the technical field of artificial intelligence. The method comprises the following steps: obtaining a first perspective image of a three-dimensional model, the first perspective image being a two-dimensional image corresponding to the three-dimensional model under a first perspective, and obtaining first category information; then determining first position information according to the first perspective image and the first category information, wherein a region indicated by the first position information in the first perspective image comprises a first region, the first region being a region corresponding to the first category information; further determining a first geometric entity corresponding to the first position information in the three-dimensional model; and finally labeling a geometric feature formed by the first geometric entity in the three-dimensional model with the first category information. Through the execution of the above method of the application, the three-dimensional model can be efficiently and accurately labeled, which helps to save manpower and reduce labeling cost.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a labeling method, apparatus, computing device cluster, and computer program product. Background Technology

[0002] With the rapid development of intelligent manufacturing, Computer-Aided Design (CAD) technology is playing an increasingly important role in the manufacturing industry, and it is widely used in various stages such as preliminary product design, production planning, quality control, and maintenance. CAD-based three-dimensional (3D) models often have a complex and numerous number of machining features, such as holes, threads, and chamfers. These features typically need to be annotated to facilitate subsequent processing of the 3D model, such as training machine learning models, helping CNC machine tools generate machining code, and simplifying the programming process.

[0003] Current annotation methods primarily rely on engineers manually annotating machining features in 3D models. This is extremely time-consuming, error-prone, and labor-intensive. While some 3D image annotation software offers intelligent suggestions to accelerate the process, these programs depend on predefined rules, which in turn rely on human experience. This makes it difficult to accurately define the machining features to be annotated, limiting their flexibility and adaptability for handling complex 3D model annotation needs, and resulting in low automation. Therefore, how to efficiently and accurately identify and annotate machining features in 3D models has become a pressing problem. Summary of the Invention

[0004] This application provides an annotation method, apparatus, computing device cluster, and computer program product that can efficiently and accurately annotate the processing features in a 3D model.

[0005] In a first aspect, this application provides an annotation method, which includes: acquiring a first-view image of a three-dimensional model, the first-view image being a two-dimensional image of the three-dimensional model from the first viewpoint, and acquiring first category information; then determining first position information based on the first-view image and the first category information, wherein the area indicated by the first position information in the first-view image includes a first region, the first region being the region corresponding to the first category information; further determining the first geometric entity corresponding to the first position information in the three-dimensional model; and then annotating the geometric features formed by the first geometric entity in the three-dimensional model with the first category information.

[0006] In the above scheme, a first-view image (which is a two-dimensional image) of the 3D model and the first category information corresponding to one type of processing feature to be labeled are obtained. Based on target detection and image segmentation techniques for two-dimensional images, the region corresponding to the first category information is located in the first-view image, thus obtaining the first position information. The region indicated by the first position information in the first-view image includes the region corresponding to the first category information. Then, the first geometric entity corresponding to the first position information in the 3D model is determined. There may be one or more first geometric entities. The geometric features formed by the first geometric entities can then be labeled with the first category information, that is, the first category information serves as a label for the geometric features formed by the first geometric entities. This label indicates that the geometric features formed by the first geometric entities belong to the processing feature corresponding to the first type of information, enabling subsequent processing of the 3D model, such as training machine learning models, helping CNC machine tools generate machining code, simplifying the programming process, etc. This scheme enables efficient and accurate labeling of specific processing features in the 3D model (specified by category information), without relying on human experience, thus saving manpower and time and reducing labeling costs.

[0007] Based on the first aspect, in possible implementations, the first location information includes a bounding box and a segmentation mask.

[0008] Based on the first aspect, in a possible implementation, the first geometric entity corresponding to the first location information in the three-dimensional model can be determined by the following steps: determining multiple geometric entities corresponding to the first location information in the three-dimensional model, then determining the centroid of each of the multiple geometric entities, and then determining the first geometric entity from the multiple geometric entities based on the centroid of each of the multiple geometric entities, wherein the point corresponding to the centroid of the first geometric entity in the first view image is located within the area indicated by the first location information in the first view image.

[0009] It should be understood that due to the polymorphism of geometric features in 3D models—for example, the shape and size of a hole may differ in different positions and directions—and because 2D images of 3D models lose more information than 3D models, the accuracy of target detection and image segmentation techniques for 2D images is limited (not guaranteed to be completely accurate), the aforementioned first position information may not be accurate enough. That is, the area indicated by the first position information in the first-view image may include not only the first region corresponding to the first category information but also regions that do not correspond to the first category information. In other words, the area indicated by the first position information may be too large, encompassing both the first region corresponding to the first category information and regions that do not correspond to the first category information. Consequently, the geometric entities projected into the 3D model based on the first position information may include both geometric entities corresponding to the first region and geometric entities that do not correspond to the first region. For ease of description, the geometric features constituted by geometric entities that do not correspond to the first region (i.e., geometric entities that do not correspond to the first category information) are called interfering geometric features. Interfering geometric features should not be labeled with the first category information; only the geometric features corresponding to the first region need to be labeled with the first category information to improve labeling accuracy.

[0010] To avoid incorrectly labeling interfering geometric features with the first category information, it is necessary to filter multiple geometric entities projected into the 3D model based on the first position information. This involves filtering out interfering geometric entities from these multiple geometric entities, and only labeling the geometric features formed by the remaining geometric entities with the first category information. In the above scheme, when multiple geometric entities are determined in the 3D model based on the first position information, the centroids of each geometric entity are calculated. Then, it is determined whether the point corresponding to the centroid of each geometric entity in the first-view image is located within the area indicated by the first position information in the first-view image, thereby filtering out geometric entities belonging to interfering geometric features. For any of the aforementioned geometric entities, if the point corresponding to the centroid of the geometric entity in the first-view image is outside the area indicated by the first position information in the first-view image, then the geometric entity is determined to be a geometric entity belonging to interfering geometric features, and thus the interfering geometric features to which the geometric entity belongs are filtered out, and the first category information is not labeled on the interfering geometric features; if the point corresponding to the centroid of the geometric entity in the first-view image is within the area indicated by the first position information in the first-view image, then the geometric entity is retained (without filtering), and the retained geometric entity is considered to be the geometric entity corresponding to the first region (i.e., the region corresponding to the first category information in the first-view image) in the 3D model, thus determining that the retained geometric entity is a geometric entity among the geometric features corresponding to the first category information in the 3D model, and the geometric features formed by the retained geometric entity can be labeled with the first category information.

[0011] Based on the first aspect, in possible implementations, the number of the aforementioned first geometric entities is multiple. An attribute adjacency graph can be constructed for these multiple first geometric entities. This attribute adjacency graph represents the topological structure of the multiple first geometric entities, facilitating storage and analysis. The attribute adjacency graph includes multiple subgraphs, each corresponding to at least one of the multiple first geometric entities. Different subgraphs correspond to different first geometric entities, and different subgraphs are divided into disjoint sets by the connected components in the attribute adjacency graph. Each subgraph represents the topological structure of its corresponding first geometric entity.

[0012] Based on the first aspect, in a possible implementation, each subgraph includes vertices and edges in the corresponding first geometric entity. The distance between each vertex in each subgraph and the observation point corresponding to the first viewpoint image can be calculated. Then, the first subgraph in the multiple subgraphs is determined based on the distance. Subsequently, the geometric features constituting the first geometric entity belonging to the first subgraph are labeled with first category information.

[0013] In other words, by calculating the distance between each vertex in each subgraph and the observation point, and then determining whether the first geometric entity corresponding to each subgraph is a geometric entity in the geometric features corresponding to the first category of information in the 3D model based on the distance between each vertex and the observation point, the annotation accuracy can be improved by filtering out geometric entities in the interfering geometric features. Here, the subgraph corresponding to the vertex with the smallest distance from the observation point can be taken as the first subgraph. The first subgraph may correspond to one or more geometric entities. The geometric features formed by the first geometric entities corresponding to the first subgraph are then determined as geometric features in the 3D model corresponding to the first category of information. The geometric features formed by the first geometric entities corresponding to the first subgraph are labeled with the first category of information. As for the geometric features formed by the corresponding geometric entities of other subgraphs besides the first subgraph, they are considered geometric features that do not correspond to the first category of information (i.e., interfering geometric features) and are not labeled with the first category of information.

[0014] Based on the first aspect, in a possible implementation, the first subgraph is the subgraph corresponding to the vertex with the smallest distance from the observation point among the vertices of multiple subgraphs.

[0015] Since the geometric entities in the geometric features corresponding to the first category of information are often positioned further forward than the geometric entities in the interfering geometric features from the first perspective, the above scheme distinguishes the geometric entities in the geometric features corresponding to the first category of information from the geometric entities in the interfering geometric features by calculating the distance from each vertex to the observation point in multiple subgraphs. The subgraph containing the vertex with the smallest distance from the observation point is taken as the first subgraph. The geometric features formed by the first geometric entities corresponding to the first subgraph are considered to be geometric features corresponding to the first category of information. The geometric features formed by the first geometric entities corresponding to other subgraphs are considered to be interfering geometric features. Only the geometric features formed by the first geometric entities corresponding to the first subgraph are labeled with the first category of information.

[0016] Based on the first aspect, in a possible implementation, each subgraph includes vertices and edges in the corresponding first geometric entity. The weight of each vertex in each subgraph of the multiple subgraphs can be calculated. The weight of each vertex is used to represent the importance of each vertex in the corresponding subgraph. Then, the target vertex of the multiple subgraphs is determined according to the weight of each vertex in the multiple subgraphs. The target vertex of any subgraph is the vertex with the highest importance in any subgraph. Then, the distance between the target vertex of the multiple subgraphs and the observation point corresponding to the first viewpoint image is calculated. The first subgraph of the multiple subgraphs is determined according to the above distance. Then, the geometric features formed by the first geometric entity belonging to the first subgraph are labeled with first category information.

[0017] In the above scheme, the target vertex of each subgraph is determined by calculating the importance of each vertex in each subgraph. The target vertex of each subgraph is the vertex with the highest importance in that subgraph. Using the highest-important vertex (target vertex) to represent the corresponding subgraph is more accurate (for subsequent distance calculations). Then, the distance between the target vertex of each subgraph and the observation point is calculated, without calculating the distance between the observation point and other vertices in each subgraph (called non-target vertices), thus eliminating the influence of non-target vertices. The geometric features formed by the first geometric entity corresponding to the first subgraph are determined as the geometric features corresponding to the first category information in the 3D model. The first category information is then labeled on the geometric features formed by the first geometric entity corresponding to the first subgraph. For the geometric features formed by the first geometric entities of other subgraphs, they are considered geometric features that do not correspond to the first category information (i.e., interfering geometric features) and are not labeled with the first category information.

[0018] Based on the first aspect, in a possible implementation, the first subgraph is the subgraph corresponding to the target vertex with the smallest distance from the observation point among the target vertices of multiple subgraphs.

[0019] Based on the first aspect, in a possible implementation scheme, the above-mentioned three-dimensional model is a computer-aided design model.

[0020] Based on the first aspect, in a possible implementation, the above-mentioned 3D model is in the Product Data Exchange Standard (PDS) format. Here, the 3D model in the DDS format is processed directly, without needing to be converted to other formats (such as point clouds, voxels, meshes, etc.) before further processing.

[0021] Secondly, this application provides an annotation device, including an acquisition module, a determination module, and an annotation module. The acquisition module is used to acquire a first-view image of a three-dimensional model, the first-view image being a two-dimensional image of the three-dimensional model from the first viewpoint. The acquisition module is also used to acquire first category information. The determination module is used to determine first position information based on the first-view image and the first category information. The area indicated by the first position information in the first-view image includes a first region, the first region being the region corresponding to the first category information. The determination module is also used to determine the first geometric entity corresponding to the first position information in the three-dimensional model. The annotation module is used to annotate the geometric features constituted by the first geometric entity in the three-dimensional model with the first category information.

[0022] The various functional modules in the above-mentioned labeling device are used in combination to implement the method of any possible implementation scheme in the first aspect, as can be seen from the above description, which will not be repeated here.

[0023] Thirdly, this application provides a computing device cluster including at least one computing device, each computing device including a processor and a memory, wherein the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform a method as described in any possible embodiment of the first aspect.

[0024] Fourthly, this application provides a computer program product including instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform a method as described in any possible embodiment of the first aspect.

[0025] Fifthly, this application provides a computer-readable storage medium including instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform a method as described in any possible embodiment of the first aspect. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments are briefly introduced below.

[0027] Figure 1 This is a flowchart illustrating an annotation method provided in an embodiment of this application;

[0028] Figure 2This is a schematic diagram of a three-dimensional model of a mechanical part provided in an embodiment of this application;

[0029] Figure 3 This is a two-dimensional image corresponding to a three-dimensional model of a mechanical part provided in an embodiment of this application from a first-view perspective;

[0030] Figure 4 This is a flowchart of annotation provided in an embodiment of this application;

[0031] Figure 5 This is a two-dimensional image of a three-dimensional model of a mechanical part provided in an embodiment of this application, viewed from a second perspective.

[0032] Figure 6 This is a flowchart of determining a first sub-graph provided in an embodiment of this application;

[0033] Figure 7 This is another flowchart for determining the first sub-graph provided in an embodiment of this application;

[0034] Figure 8 This is a schematic diagram of the structure of a labeling device provided in an embodiment of this application;

[0035] Figure 9 This is a schematic diagram of another labeling device provided in an embodiment of this application;

[0036] Figure 10 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0037] Figure 11 This is a schematic diagram of a computing device cluster provided in an embodiment of this application;

[0038] Figure 12 This is a schematic diagram illustrating the connection between two computing devices via a network, as provided in an embodiment of this application. Detailed Implementation

[0039] To facilitate understanding of the technical solutions in the embodiments of this application, some terms and concepts involved in the embodiments of this application will be briefly introduced first.

[0040] 1. Standard for the Exchange of Product Model Data (STEP): This is a 3D model data format developed by the International Organization for Standardization (ISO). Its file extension is ".step" or ".stp". It is mainly used for data exchange and sharing between different CAD / Computer-Aided Manufacturing (CAM) software. It not only supports the description of geometric shapes (geometric entities) but also includes rich information such as precise dimensional parameters. It is widely used in engineering design and manufacturing fields.

[0041] 2. Attribute Adjacency Graph (AAG): This is a graph structure used to describe and analyze the relationships between concepts. In fields such as engineering design and manufacturing, 3D models can be converted into corresponding attribute adjacency graphs to describe the model's topology and attribute information (such as location coordinates, color, texture, etc.). The definition of the attribute adjacency graph G can be found in Equation (1):

[0042] G = {V, E} (1)

[0043] Points, lines, and surfaces are geometric elements (geometric entities) in the model. V is the set of vertices in the attribute adjacency graph, corresponding to the surfaces in the model. E is the set of edges in the attribute adjacency graph, corresponding to the edges formed by the intersection of two surfaces in the model (i.e., the common edges of adjacent surfaces). Surfaces are enclosed by edges, and edges can also reflect the connection relationship between vertices.

[0044] 3. Page Rank (Pr) score: This is one of the important standards for measuring the quality of a website. Its physical meaning is the probability that a website will be visited. It evaluates the importance of each node (i.e., webpage) by analyzing the network graph (i.e., the hyperlink structure between webpages). Pr scores range from 0 to 10, with 10 being the highest score. The higher the Pr score, the more popular (more important) the webpage. Besides network graphs, Pr scores can also be used to evaluate the importance (or influence) of nodes in other types of graphs. For example, if a vertex connects to many other important vertices, then that vertex itself is considered important.

[0045] 4. Boundary Representation: This is a geometric modeling technique used in CAD, computer graphics, and solid modeling. It describes three-dimensional solids by defining the surfaces or boundaries of an object.

[0046] Based on the above, embodiments of the annotation method provided in this application are described below.

[0047] Please see Figure 1 , Figure 1 This is a flowchart illustrating an annotation method provided in an embodiment of this application, including steps S101 to S105.

[0048] S101. Obtain the first-view image of the 3D model. The first-view image is the 2D image of the 3D model from the first perspective.

[0049] The aforementioned 3D model can be a model designed based on a CAD system, and its format can be STEP format. This application does not specifically limit the content described by the 3D model. For example, the 3D model may be used to describe the geometric structure, dimensional parameters, and other physical properties (such as material, density, color, etc.) of a machined part / component / equipment. This application also does not specifically limit the source of the 3D model. For example, users can input custom 3D models for execution. Figure 1 This method can also obtain 3D models from CAD systems, other systems, or platforms for execution. Figure 1 The method.

[0050] The aforementioned first-viewpoint is the perspective from which a 3D model is observed. The first-viewpoint image is a two-dimensional (2D) image captured from an observation point at a distance r from the origin of the coordinate system to which the 3D model belongs. The first-viewpoint has a corresponding viewing direction, which includes the elevation angle θ and the azimuth angle ψ.

[0051] It should be understood that r, θ, and ψ together determine the position of the observation point relative to the origin of the coordinate system. r represents the distance (radial length / linear distance) between the observation point and the origin of the coordinate system, and its value is a non-negative real number (r≥0). θ represents the angle between the observation point and the polar axis of the coordinate system (e.g., the z-axis), and its value is 0≤θ≤π. When θ=0, the observation point is located in the positive direction of the polar axis; when θ=π, the observation point is located in the negative direction of the polar axis. ψ represents the angle of rotation of the observation point around the polar axis in a plane perpendicular to the polar axis (e.g., the plane containing the x-axis and y-axis), and its value is 0≤ψ<2π. This application does not specifically limit the values ​​of r, θ, and ψ; users can set them according to their needs. They can be default values ​​or generated according to a certain algorithm.

[0052] In addition to obtaining the 2D image of the 3D model from the first perspective, it is also possible to obtain the 2D image of the 3D model from other perspectives (one or more), thus obtaining multiple perspective images of the 3D model. Each perspective image corresponds to a perspective, and the position of the observation point corresponding to each perspective image is determined by the corresponding r, θ, and ψ.

[0053] For example, the position, intensity, and type of the light source in the virtual environment where the 3D model is located can be set. Then, a virtual camera can be used to capture 2D images of the 3D model in the virtual environment from multiple perspectives, thereby obtaining multiple perspective images of the 3D model. To facilitate subsequent processing of the images from each perspective, operations such as resolution adjustment and noise reduction can be performed on the images from each perspective to improve image quality, so that the region corresponding to the specified category information can be located more accurately in each perspective image (see the description in step S103).

[0054] S102, Obtain the information for the first category.

[0055] The first category of information consists of information on one type of processing feature that needs to be identified and labeled (such as name, number, feature description, etc.). Processing features, sometimes also called manufacturing features, refer to the shape characteristics of materials after cutting and shaping, i.e., geometric features with processing semantics. This application does not specifically limit the types of processing features corresponding to the first category of information; the classification of processing feature types can be adjusted according to the needs of actual application scenarios. For example, it can include geometric features such as chamfered corners, through holes, triangular channels, rectangular channels, circular through slots, rectangular through steps, O-rings, blind holes (blind holes), triangular end pockets, rectangular blind slots, T-slots, triangular blind steps, and wire grooves.

[0056] This application does not specifically limit the source of the first category information. For example, the first category information can be set by the user, that is, the user can specify the information of the processing features that need to be identified and labeled. Regarding the setting of the first category information, the user can directly input the first category information, or the user can select one of multiple existing category information (such as information provided by the labeling device) as the first category information. In practical application scenarios, in addition to setting the first category information, the user can also set other types of processing feature (one or more) information as needed, thereby obtaining multiple categories of information, each type of information corresponding to one type of processing feature.

[0057] This application does not specify the execution order of steps S101 and S102. For example, step S101 can be executed first and then step S102, or step S102 can be executed first and then step S101, or steps S101 and S102 can be executed in parallel.

[0058] S103. Determine first location information based on the first viewpoint image and the first category information. The area indicated by the first location information in the first viewpoint image includes a first region, which is the region corresponding to the first category information.

[0059] As introduced above, a first-view image is a two-dimensional image corresponding to a three-dimensional model from the first perspective. Therefore, techniques such as object detection and image segmentation for two-dimensional images can be used to determine the location information of regions corresponding to the first category information in the first-view image. One or more regions in the first-view image may correspond to the first category information, and the first region is one of these regions. This application does not specifically limit the object detection and image segmentation techniques mentioned above. For example, image segmentation may include semantic segmentation and instance segmentation.

[0060] The region indicated by the aforementioned first location information in the first-view image includes the first region corresponding to the first category information. The first location information may include a bounding box and a segmentation mask. The bounding box is sometimes also called a localization box, and the segmentation mask is sometimes called a mask. The segmentation mask is used to indicate which pixels in the first-view image belong to the region indicated by the first location information. This application does not specifically limit the type of bounding box. For example, the bounding box can be a rectangle, which is a minimum bounding rectangle of the aforementioned region, used to roughly locate the boundary of the aforementioned region in the first-view image. Besides rectangles, the bounding box can also be other shapes (such as other polygonal boxes) to more accurately describe the boundary of the aforementioned region in the first-view image.

[0061] For example, such as Figure 2 As shown, assuming the object to be annotated is a 3D model of a mechanical part, step S101 obtains a 2D image of the 3D model of the mechanical part from a first-view perspective (see...). Figure 3 The image obtained in step S102 is a first-view image, and the first category information is "hole," indicating that processing features such as "holes" in the 3D model need to be labeled. Then, step S103 determines the location information (including bounding boxes and corresponding segmentation masks) of the regions corresponding to "holes" in the first-view image. Figure 3As shown, there are four regions corresponding to the holes in the first-view image. For ease of description and differentiation, these four regions are referred to as A, B, C, and D. Each region has a corresponding bounding box (represented by a rectangular dashed box) and a segmentation mask (represented by a gray fill). The bounding box (here, a rectangle) is used to indicate the boundary of the corresponding region, and the segmentation mask is used to indicate which pixels in the first-view image belong to the region corresponding to the segmentation mask.

[0062] Similarly, we can also obtain the corresponding two-dimensional images of the above three-dimensional model from other perspectives (one or more perspectives), obtain one or more perspective images, and then determine the location information of the region corresponding to the first category information in each perspective image. This will not be discussed in detail here.

[0063] S104. Determine the first geometric entity in the three-dimensional model corresponding to the first position information.

[0064] As introduced above, the first-view image of a 3D model has corresponding r, θ, and ψ values. Therefore, based on the corresponding r, θ, and ψ values ​​of the first-view image, the geometric entity corresponding to the first position information in the 3D model can be determined. The first position information may correspond to one or more geometric entities in the 3D model. The first geometric entity belongs to one or more of the aforementioned geometric entities, and there may be one or more first geometric entities. The first geometric entity may be all of the aforementioned one or more geometric entities, or it may be some of the aforementioned one or more geometric entities.

[0065] The geometric entity can be a point, line, or surface, etc. For example, the geometric entity can be a Cartesian point, branch, ellipse, parabola, polyline, composite curve, B-spline curve, plane, conical surface, toroidal surface, linearly stretched surface, B-spline surface, offset surface, etc. This application does not make specific limitations on this.

[0066] In step S103, regarding Figure 3 For example, the two-dimensional image (i.e., the first-view image) corresponding to the three-dimensional model of a mechanical part in the first-view perspective includes four regions corresponding to "holes", which are denoted as regions A, B, C, and D respectively.

[0067] Taking region A as an example, the location information of region A includes the bounding box and segmentation mask corresponding to region A in the first-view image. The corresponding bounding box of region A (represented by a rectangular dashed box) indicates the boundary of region A, and the corresponding segmentation mask of region A (the gray-filled part in the bounding box) indicates which pixels in the first-view image belong to region A. Based on the r, θ, and ψ values ​​corresponding to the first-view image, the segmentation mask of region A can be projected onto the geometric entities of the 3D model using ray projection. The geometric entities onto which the segmentation mask of region A is projected on the 3D model may be one or more.

[0068] As introduced above, the first geometric entity is one of the geometric entities corresponding to the first location information in the 3D model. There may be one or more first geometric entities, which may be some or all of the geometric entities corresponding to the first location information in the 3D model. Here, an attribute adjacency graph (AAG) can be constructed for multiple first geometric entities. The attribute adjacency graph is used to represent the topological structure of these multiple first geometric entities. The attribute adjacency graph includes multiple subgraphs, each subgraph corresponding to at least one of the multiple first geometric entities. Different subgraphs correspond to different first geometric entities. Different subgraphs are disjoint sets divided by connected components in the attribute adjacency graph. Each subgraph is used to represent the topological structure (connection relationship) of the corresponding first geometric entity. If the above attribute adjacency graph is a connected graph, then the attribute adjacency graph has only one connected component, which is itself. If the above attribute adjacency graph is a disconnected undirected graph, then the attribute adjacency graph has multiple connected components. Each geometric entity within a subgraph intersects with at least one geometric entity within the same subgraph. Geometric entities from different subgraphs do not intersect, and corresponding geometric entities within the same subgraph collectively constitute a geometric feature. It should be understood that attribute adjacency graphs can accurately abstract the topological relationships of geometric entities (such as points, lines, and surfaces) within geometric features, providing a concise, easy-to-store, and easy-to-analyze data format.

[0069] It should be noted that due to the polymorphism of geometric features in 3D models—for example, the shape and size of a hole may differ in different positions and directions—and because 2D images of 3D models lose more information than 3D models, the accuracy of target detection and image segmentation techniques for 2D images is limited (not guaranteed to be completely accurate), the first position information determined in step S103 may not be accurate enough. That is, the area indicated by the first position information in the first view image may include not only the first region corresponding to the first category information but also regions that do not correspond to the first category information. In other words, the area indicated by the first position information is too large, covering both the first region corresponding to the first category information and regions that do not correspond to the first category information. Consequently, the geometric entities projected into the 3D model based on the first position information (at this time, there are multiple geometric entities) include geometric entities corresponding to the first region and geometric entities that do not correspond to the first region (i.e., geometric entities that do not correspond to the first category information). For ease of description, the geometric features constituted by geometric entities that do not correspond to the first region are called interfering geometric features. Interfering geometric features should not be labeled with the first category information; only the geometric features corresponding to the first region need to be labeled with the first category information to improve the labeling accuracy.

[0070] To avoid incorrectly labeling interfering geometric features with the first category information, it is necessary to filter the multiple geometric entities projected into the 3D model based on the first position information. The geometric entities that belong to the interfering geometric features are filtered out from these multiple geometric entities, and the first category information is only labeled for the geometric features formed by the remaining geometric entities among these multiple geometric entities, without labeling the interfering geometric features with the first category information.

[0071] In one possible implementation, the first geometric entity can be determined using the following centroid filtering mechanism: First, determine multiple geometric entities corresponding to the first position information in the 3D model. Then, determine the centroid (center of mass) of each of these multiple geometric entities. The centroid, sometimes called the mass point, is an imaginary point where the mass of the geometric entity is considered to be concentrated. In other words, the centroid is a simplified geometric point that only has mass and disregards size and shape. This application does not specifically limit the method for determining the centroid of the geometric entity; appropriate algorithms can be used as needed. Next, determine the first geometric entity (which may have one or more) from these multiple geometric entities based on the centroid of each of them. The point corresponding to the centroid of the first geometric entity in the first-view image is located within the area indicated by the first position information in the first-view image.

[0072] In other words, when multiple geometric entities are determined in the 3D model based on the first position information, the centroids of these geometric entities can be calculated. Then, it can be determined whether the point corresponding to the centroid of each geometric entity in the first-view image is located within the area indicated by the first position information in the first-view image, thereby filtering out geometric entities belonging to interfering geometric features. For any geometric entity among the multiple geometric entities, if the point corresponding to the centroid of the geometric entity in the first-view image is located outside the area indicated by the first position information in the first-view image, then the geometric entity is determined to be a geometric entity belonging to interfering geometric features, and thus the interfering geometric features to which the geometric entity belongs are filtered out, and the first category information will not be labeled for the interfering geometric features; if the point corresponding to the centroid of the geometric entity in the first-view image is located within the area indicated by the first position information in the first-view image, then the geometric entity is retained (without filtering). It is considered that the retained geometric entity is the geometric entity corresponding to the first region (i.e., the region corresponding to the first category information in the first-view image) in the 3D model, and thus it can be determined that the retained geometric entity is a geometric entity among the geometric features corresponding to the first category information in the 3D model, and the geometric features constituted by the retained geometric entities can be labeled with the first category information.

[0073] In step S103, regarding Figure 3For example, region C in the two-dimensional image is enclosed by a rectangular bounding box, which is used to indicate that the region inside contains the corresponding first category information. The region indicated by the corresponding segmentation mask in the bounding box is represented by gray fill. The region indicated by the segmentation mask contains the region corresponding to the first category information, and may also contain regions that do not correspond to the first category information.

[0074] Will Figure 3 The segmentation mask of the bounding box in the above image is projected onto... Figure 2 On the 3D model, two geometric entities can be identified. The first geometric entity is surface 1 in the 3D model (represented by a diagonal fill), and the second geometric entity is plane 2 in the 3D model (represented by a gray fill). It should be noted that although surface 1 and plane 2 do not overlap on the 3D model, the corresponding regions of surface 1 and plane 2 overlap in the first-view image. This makes the segmentation mask obtained when using image segmentation techniques for 2D images to segment the first-view image inaccurate. The region indicated by the segmentation mask includes both the region corresponding to surface 1 and a portion of the region corresponding to plane 2. The geometric entities identified by projecting this segmentation mask onto the 3D model include plane 2 and surface 1. The geometric features formed by plane 2 interfere with the geometric features formed by the labeled surface 1.

[0075] Here, we can calculate the centroids of surface 1 and plane 2, and project their centroids onto... Figure 3 In the 2D image (i.e., the first-view image) from this perspective, since the centroid of surface 1 (not directly marked here) corresponds to a point (not directly marked here) in the first-view image that is located within the area indicated by the segmentation mask, surface 1 can be considered a geometric entity in the geometric features corresponding to the first category of information in the 3D model. Therefore, the geometric features formed by surface 1 can be labeled with the first category of information. Similarly, since the centroid of plane 2 (not directly marked here) corresponds to a point (not directly marked here) in the first-view image that is located outside the area indicated by the segmentation mask, plane 2 can be determined as the geometric entity in the 3D model corresponding to the region in the first-view image that does not correspond to the first category of information. That is, plane 2 is determined to be a geometric entity in the interfering geometric features. Therefore, plane 2 is excluded, and the geometric entities formed by plane 2 will not be labeled with the first category of information in the first view, thereby improving the labeling accuracy.

[0076] For example, Figure 4 This is a labeling flowchart provided in an embodiment of the present application, including steps S401 to S406.

[0077] S401. Input a 3D model in step format.

[0078] For information on the STEP format and 3D models, please refer to the previous introductions; they will not be repeated here.

[0079] It should be understood that 3D models contain a wealth of information. To facilitate subsequent processing, the 3D model in step format can be preprocessed to filter information related to geometric entities, as well as dimensional parameters and other physical properties (such as density, material, color, etc.).

[0080] S402. Use a virtual camera to select k viewpoints and render k viewpoint images (2D images) of the 3D model.

[0081] Specifically, a virtual camera can be used to capture 2D images corresponding to the 3D model from k viewpoints, resulting in k viewpoint images, and the corresponding viewpoint direction of each viewpoint image can be recorded. Resolution adjustment, noise reduction, and other optimization processes can also be performed on each viewpoint image to improve image quality, which helps improve the subsequent image segmentation results.

[0082] The above k is a positive integer greater than 1. The value of k can be set by the user according to the actual usage needs. For example, the user can set k=7, or it can be the default value. This application does not make any specific restrictions on this.

[0083] S403. Obtain m category information, perform image segmentation on the images from each viewpoint, and obtain n bounding boxes and n segmentation masks.

[0084] The value of m is a positive integer, and can be set according to usage needs. This application does not impose any specific limitations on it. For example, a user can input m text prompts as m category information. Each category information is information about a type of processing feature that needs to be labeled. For information on processing features and category information, please refer to the previous introduction, which will not be repeated here.

[0085] The above n bounding boxes correspond one-to-one with n segmentation masks, each bounding box corresponding to a category. For details on bounding boxes and segmentation masks, please refer to the previous sections; they will not be repeated here. Regarding how to segment the images from each viewpoint based on the category information to determine the corresponding bounding boxes and segmentation masks in each viewpoint image, please refer to the previous sections; they will not be repeated here. The above n can be greater than or equal to the above m. For any category information, there may be one or more bounding boxes corresponding to that category information determined in these k viewpoint images.

[0086] S404. Project the segmentation mask of n bounding boxes onto the 3D model to obtain a set of geometric entities.

[0087] For instructions on how to project the segmentation mask of a bounding box onto a 3D model, please refer to the previous section; details will not be repeated here. Each bounding box's segmentation model can be projected onto one or more geometric entities on the 3D model. The aforementioned set of geometric entities is the collection of geometric entities projected onto the 3D model from the segmentation masks of n bounding boxes. Geometric entities can be points, lines, faces, etc., and the set of these geometric entities can be denoted as (entity1, entity2,...), where entity represents a geometric entity, and the number after entity is used to distinguish different geometric entities. This representation is merely an example and does not constitute a limitation.

[0088] S405. Perform operations for each bounding box, including steps S405_1 to S405_3.

[0089] S405_1. Calculate the centroid of each geometric entity onto which the segmentation mask of the bounding box is projected, and obtain the centroid set.

[0090] Specifically, the segmentation mask of a bounding box can be projected onto one or more geometric entities on a 3D model. The centroids of these one or more geometric entities are calculated to obtain a set of centroids, which can be denoted as Mass = {mass1, mass2, ...}, where mass represents the centroid. Each centroid corresponds to a geometric entity on the 3D model, and the number after mass is used to distinguish different centroids in the set. This representation is only an example and does not constitute a limitation.

[0091] S405_2: Select centroids that meet the set conditions from the set of centroids.

[0092] The above setting condition is that the point corresponding to the centroid in the corresponding 2D image of this bounding box is located within the area indicated by the segmentation mask of this bounding box.

[0093] For example, suppose "hole" is one of the categories of information input by the user, such as Figure 3 As shown, Figure 3 It is a two-dimensional image (i.e., first-view image) corresponding to the three-dimensional model of the mechanical part from the first-view perspective. Based on this category information, four regions are determined in the first-view image as position information. These four regions are denoted as regions A, B, C, and D, and each region includes the region corresponding to the "hole".

[0094] Taking region A as an example, the location information of region A includes the bounding box that selects the region (represented by a dashed rectangle) and the segmentation mask within the bounding box (represented by a gray fill). The segmentation mask of the bounding box is projected onto... Figure 2 On the 3D model, the geometric entity of surface 3 (the inner wall of the hole) can be determined. Assume the centroid of surface 3 (not directly labeled here) is located at... Figure 3If the corresponding point in the 2D image (not directly marked here) is located within the area indicated by the segmentation mask mentioned above, then surface 3 can be considered a geometric entity in the 3D model corresponding to the category information of "hole". Therefore, the geometric features formed by surface 3 can be labeled with the first category information. Similarly, the geometric features formed by the corresponding geometric entities in other regions of the 3D model can be labeled with the corresponding category information.

[0095] For example, Figure 5 This is a two-dimensional image of a three-dimensional model of a mechanical part provided in an embodiment of this application, viewed from a second perspective, and denoted as the second-view image. The second perspective is different from... Figure 3 The first-view image corresponds to the first-view perspective. Assuming that "hole" is one of the category information input by the user, target detection and image segmentation are performed on the second-view image based on this category information. The location information of two regions can be determined in the second-view image, denoted as region E and region F respectively. Both regions contain the region corresponding to the "hole".

[0096] Taking region E as an example, the location information of region E includes the bounding box (represented by a dashed rectangle) and the segmentation mask within the bounding box (represented by a thick solid line). The segmentation mask of the bounding box is projected onto... Figure 2 On the 3D model, the geometric entity of surface 4 (the inner wall of the hole) can be determined, and then the centroid of surface 4 can be calculated. For example... Figure 5 As shown, since the point corresponding to the centroid of surface 4 in the second-view image (represented by the symbol ×) is outside the region indicated by the segmentation mask of the bounding box (i.e., region E), the centroid does not meet the set conditions. This indicates that the geometric entity corresponding to the centroid is occluded (covered) by other geometric entities in the second-view image. Therefore, it is not suitable to label the geometric features formed by surface 4 in the 3D model as "holes" in the second-view image. Similarly, the segmentation mask corresponding to region E is projected onto... Figure 2 On the 3D model, the geometric entity of surface 5 (the inner wall of the hole) can be determined, and then the centroid of surface 5 can be calculated. Since the point corresponding to the centroid of surface 5 in the second-view image (represented by the symbol ×) is outside the area indicated by the above segmentation mask (i.e., region F), the centroid does not meet the set conditions. This means that the geometric entity corresponding to the centroid is occluded by other geometric entities in the second view. Therefore, it is not suitable to label the geometric feature formed by surface 5 in the 3D model as a "hole" in the second view.

[0097] S405_3. Construct an attribute adjacency graph for the geometric entities corresponding to the centroids that meet the set conditions.

[0098] It should be understood that for a bounding box, its segmentation mask may project onto one or more geometric entities in the 3D model. By calculating the centroids of these geometric entities, it is determined which geometric entities' centroids meet certain conditions (see previous description). There may be one or more centroids that meet the conditions, or none at all. For the bounding box, if there are multiple centroids that meet the conditions, an attribute adjacency graph can be constructed for the geometric entities corresponding to these multiple compliant centroids (as the first geometric entity). The attribute adjacency graph is used to indicate the topological structure of the geometric entities corresponding to these multiple compliant centroids. The attribute adjacency graph includes multiple subgraphs, each corresponding to at least one geometric entity among the multiple compliant centroids. Different subgraphs correspond to different geometric entities, and different subgraphs are divided into disjoint sets by the connected components in the attribute adjacency graph. Each subgraph represents the topological structure of the corresponding geometric entity.

[0099] S406. Label the corresponding category information for the geometric features formed by the geometric entities selected by each bounding box.

[0100] As introduced above, the segmentation mask of a bounding box may correspond to one or more geometric entities on a 3D model. These one or more geometric entities are filtered according to preset conditions, and the geometric entities that meet the preset conditions are taken as the geometric entities filtered by the bounding box. Each bounding box may have no geometric entities filtered (in which case no label is made), or it may have one or more. In this case, the geometric features formed by the filtered one or more geometric entities can be labeled with the category information corresponding to the bounding box.

[0101] S105. Label the first category information of the geometric features formed by the first geometric entity in the three-dimensional model.

[0102] It should be understood that the geometric features formed by the first geometric entities (which may be one or more) in the 3D model are labeled with first type information, that is, the first category information is used as the label of the geometric features formed by these one or more first geometric entities. The label indicates that the geometric features formed by these one or more first geometric entities belong to the type of machining features corresponding to the first type information, so as to enable subsequent processing of the 3D model, such as training machine learning models, helping CNC machine tools generate machining code, simplifying the programming process, etc., without specific limitations.

[0103] It should be noted that, Figure 1The method uses first-view and first-category information as an example to describe how to annotate the geometric features constituting the first geometric entity in the 3D model with first-category information based on a 2D image (i.e., first-view image) of the 3D model from the first perspective. Similarly, other category information (one or more) besides the first-category information can be obtained, and then the geometric features in the 3D model corresponding to other category information can be annotated accordingly based on the first-view image. In practical application scenarios, one can also refer to... Figure 1 The method obtains the corresponding two-dimensional images of the three-dimensional model from other perspectives (one or more), and then labels the geometric features of the three-dimensional model with the corresponding category information based on the two-dimensional images of the three-dimensional model from other perspectives.

[0104] In one possible implementation, the attribute adjacency graph includes multiple subgraphs (all of which can be referred to in the previous description). All first geometric features in the attribute adjacency graph may be all geometric features corresponding to the first position information in the 3D model, or they may be some geometric features corresponding to the first position information in the 3D model (for example, some geometric features are filtered out by a centroid filtering mechanism, and what remains after filtering are the first geometric features). Each subgraph includes vertices and edges in the corresponding first geometric entity. The distance between each vertex in each subgraph and the observation point corresponding to the first view image can be calculated. Then, the first subgraph in the multiple subgraphs is determined based on the above distance, and the geometric features composed of the first geometric entities in the 3D model belonging to the first subgraph are labeled with the first category information. The first subgraph can be the subgraph corresponding to the vertex with the smallest distance to the observation point among the vertices of the multiple subgraphs. Alternatively, the first subgraph can be the subgraph corresponding to the vertices of the multiple subgraphs whose distance to the observation point is less than or equal to a distance threshold. The distance threshold can be set according to the actual application scenario, and this application does not specifically limit it.

[0105] In other words, the labeling accuracy can be improved by calculating the distance between each vertex in each subgraph and the observation point, and then determining whether the first geometric entity corresponding to each subgraph is the first geometric entity in the 3D model corresponding to the first category information based on the distance between each vertex and the observation point. Here, the subgraph corresponding to the vertex with the smallest distance from the observation point can be taken as the first subgraph, and then the first geometric entity corresponding to the first subgraph can be determined as the first geometric entity in the 3D model corresponding to the first category information. Then, the geometric features formed by the first geometric entities (which may be one or more) corresponding to the first subgraph are labeled with the first category information. As for the geometric features formed by the first geometric entities of other subgraphs besides the first subgraph, they are considered to be geometric features that do not correspond to the first category information (i.e., interfering geometric features) and will not be labeled with the first category information.

[0106] For example, Figure 6This application provides a flowchart for determining a first subgraph, including steps S601 to S605.

[0107] S601. Construct an attribute adjacency graph, which includes multiple subgraphs.

[0108] For information on attribute adjacency graphs and subgraphs, please refer to the previous introduction; it will not be repeated here.

[0109] S602. Filter the vertices in multiple subgraphs to obtain a vertex set.

[0110] As introduced above, a subgraph includes the vertices and edges of the corresponding first geometric entity. Here, we filter the vertices in multiple subgraphs to obtain a vertex set, denoted as Vertex = (vertex1, vertex2, ...), where vertex represents a vertex, and the number after vertex is used to distinguish different vertices. This representation is only an example and does not constitute a specific limitation.

[0111] S603. Calculate the distance between each vertex in the vertex set and the observation point.

[0112] S604. Select the subgraph corresponding to the vertex with the smallest distance from the observation point among the vertices of multiple subgraphs as the first subgraph.

[0113] As introduced above, the first-view image is a two-dimensional image captured from the observation point at a distance r from the origin of the coordinate system to which the 3D model belongs, viewed from the first perspective. The first perspective has a corresponding viewing direction, including the elevation angle θ and the azimuth angle ψ. Since the position of the geometric features corresponding to the first category of information is often closer to the position of the interfering geometric features under the first perspective, the geometric entities in the geometric features corresponding to the first category of information and the geometric entities in the interfering geometric features can be distinguished by calculating the distance from each vertex in the vertex set to the observation point. The subgraph containing the vertex with the smallest distance from the observation point is taken as the first subgraph. The first geometric entity corresponding to the first subgraph is considered to be the first geometric entity corresponding to the first category of information. The geometric features formed by the first geometric entities corresponding to the first subgraph are determined to correspond to the first category of information. The first geometric entities corresponding to other subgraphs are considered to be geometric entities in the interfering geometric features. Only the geometric features formed by the first geometric entities corresponding to the first subgraph are labeled with the first category of information.

[0114] For example, the attribute adjacency graph can be denoted as graph, graph = (g1, g2, ...), where g represents a subgraph, and the number after g is used to distinguish different subgraphs in graph. The distance from vertex j in subgraph gi to the observation point is denoted as dij, i∈[1,k], and k is the number of subgraphs. The above representation is only an example and does not constitute a limitation.

[0115] S605. Label the geometric features formed by the first geometric entity corresponding to the first subgraph with the first category information.

[0116] In another possible implementation, the attribute adjacency graph includes multiple subgraphs (all described above). Each subgraph includes vertices and edges in the corresponding first geometric entity. The weight of each vertex in each subgraph can be calculated, representing its importance (importance value / importance score) within the corresponding subgraph. Then, target vertices of the multiple subgraphs are determined based on their weights. The target vertex of any subgraph is the vertex with the highest importance in that subgraph. The distance between the target vertices of the multiple subgraphs and the observation point corresponding to the first viewpoint image is calculated. Based on this distance, the first subgraph is determined, and the geometric features formed by the first geometric entity belonging to the first subgraph in the 3D model are labeled with first category information. The first subgraph can be the subgraph corresponding to the target vertex with the smallest distance to the observation point among the target vertices of the multiple subgraphs. Alternatively, the first subgraph can be the subgraph corresponding to the target vertices of the multiple subgraphs whose distance to the observation point is less than or equal to a distance threshold. The distance threshold can be set according to the actual application scenario, and this application does not specifically limit it.

[0117] In other words, the target vertex of each subgraph can be determined by calculating the importance of each vertex in each subgraph. The target vertex of each subgraph is the vertex with the highest importance in that subgraph. Using the highest-important vertex (target vertex) to represent the corresponding subgraph is more accurate (for subsequent distance calculations). Then, the distance between the target vertex of each subgraph and the observation point is calculated, without calculating the distance between the observation point and other vertices in each subgraph (called non-target vertices), to eliminate the influence of non-target vertices. Here, the subgraph corresponding to the target vertex with the smallest distance to the observation point can be taken as the first subgraph. The first geometric entity corresponding to the first subgraph is then determined as the first geometric entity in the 3D model corresponding to the first category information. The geometric features formed by the first geometric entity corresponding to the first subgraph are then labeled with the first category information. The geometric features formed by the first geometric entities of other subgraphs are considered geometric features that do not correspond to the first category information (i.e., interfering geometric features) and are not labeled with the first category information.

[0118] Optionally, the aforementioned weights can be Pr values. For more information on Pr values, please refer to the previous section; details will not be repeated here. In practical applications, besides Pr values, other values ​​that can measure the importance of a vertex can also be calculated as the aforementioned weights. For example, the number of connections (i.e., degree, representing the number of edges connected to the vertex) of a vertex in its corresponding subgraph, the average distance between a vertex and all other vertices in its corresponding subgraph, etc., are not specifically limited in this application.

[0119] For example, Figure 7 This is another flowchart for determining the first sub-graph provided in the embodiments of this application, including steps S701 to S705.

[0120] S701. Construct an attribute adjacency graph, which includes multiple subgraphs.

[0121] For information on attribute adjacency graphs and subgraphs, please refer to the previous introduction; it will not be repeated here.

[0122] S702. Filter the vertices in multiple subgraphs to obtain a vertex set.

[0123] S703. Calculate the weight of each vertex in the vertex set.

[0124] The weight of each vertex is used to represent the importance of that vertex in the corresponding subgraph. For example, the Pr value can be calculated. Please refer to the previous introduction for details, which will not be repeated here.

[0125] S704. Determine the target vertex of multiple subgraphs based on the weight of each vertex in the multiple subgraphs.

[0126] The target vertex of any subgraph can be the vertex with the highest importance in that subgraph. For example, assuming there are k subgraphs, we can calculate the Pr value of each vertex in these k subgraphs, and then determine a target vertex for each subgraph based on the Pr value of each vertex in the k subgraphs, thus obtaining k target vertices. The set of the k target vertices is Pr_max, Pr_max = (pr1, pr2, ..., prk), where pr is the Pr value of the vertex with the largest Pr value in the i-th subgraph, i ∈ [1, k], and k is the number of subgraphs.

[0127] S705. Calculate the distance between the target vertex and the observation point in each of the multiple subgraphs.

[0128] Continuing from the example in the previous step, for a set of k target vertices, Pr_max = (pr1, pr2, ..., prk), calculate the distance between each target vertex in the set and the observation point. Regarding the observation point, please refer to the previous introduction; it will not be repeated here.

[0129] S706. Select the subgraph corresponding to the target vertex with the smallest distance from the observation point among the target vertices of multiple subgraphs as the first subgraph.

[0130] S707. Label the geometric features formed by the first geometric entity corresponding to the first subgraph with the first category information.

[0131] In summary, the annotation method provided in this application obtains a first-view image (which is a two-dimensional image) of the 3D model and the first category information corresponding to the processing feature type to be annotated. Based on target detection and image segmentation techniques for two-dimensional images, the region corresponding to the first category information is located in the first-view image to obtain first position information. The region indicated by the first position information in the first-view image includes the region corresponding to the first category information. Then, the first geometric entity corresponding to the first position information in the 3D model is determined, and the geometric features constituted by the first geometric entity are annotated with the first category information. To improve annotation accuracy, the centroid of the geometric entity corresponding to the first position information in the 3D model (i.e., centroid filtering mechanism) and the distance between vertices and observation points in the subgraph can be calculated to filter out geometric entities in interfering geometric features. For details, please refer to the preceding description; further elaboration is not provided here. The method in this application enables efficient and accurate annotation of processing features in a 3D model, without relying on manual experience, thus saving manpower and reducing annotation costs.

[0132] It should be noted that, Figure 1 Besides identifying and annotating specific processing features (specified by category information) in 3D models, this method can also be used for scenarios such as model completion and style control. For example, it can be used through... Figure 1 Steps S101 to S104 locate the processing features that need to be controlled. The processing features that need to be controlled are declared through the first category information in step S102. That is, the first category information in this scenario is the information of a type of processing feature that needs to be controlled. Then, the located processing features can be subjected to style control, model completion and other operations as needed.

[0133] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a labeling device 800 provided in an embodiment of this application, including an acquisition module 801, a determination module 802 and a labeling module 803.

[0134] The acquisition module 801 is used to acquire a first-view image of the 3D model. The first-view image is a 2D image of the 3D model from the first viewpoint. The acquisition module 801 is also used to acquire first category information.

[0135] The determining module 802 is used to determine the first position information based on the first view image and the first category information. The area indicated by the first position information in the first view image includes a first region, which is the region corresponding to the first category information. The determining module 802 is also used to determine the first geometric entity corresponding to the first position information in the three-dimensional model.

[0136] The annotation module 803 is used to annotate the first category information of the geometric features constituted by the first geometric entity in the 3D model.

[0137] Optionally, the aforementioned first location information includes a bounding box and a segmentation mask.

[0138] Optionally, the determining module 802 is specifically used to: determine multiple geometric entities corresponding to the first position information in the three-dimensional model; determine the centroid of each of the multiple geometric entities; determine the first geometric entity from the multiple geometric entities based on the centroid of each of the multiple geometric entities, wherein the point corresponding to the centroid of the first geometric entity in the first view image is located within the area indicated by the first position information in the first view image.

[0139] Optionally, the number of first geometric entities is multiple, and the determining module 802 is further configured to: construct an attribute adjacency graph for the multiple first geometric entities, wherein the attribute adjacency graph is used to represent the topological structure of the multiple first geometric entities, the attribute adjacency graph includes multiple subgraphs, each subgraph corresponds to at least one of the multiple first geometric entities, different subgraphs correspond to different first geometric entities, different subgraphs are divided into disjoint sets by the connected components in the attribute adjacency graph, and each subgraph is used to represent the topological structure of the corresponding first geometric entity.

[0140] Optionally, each subgraph includes vertices and edges in the corresponding first geometric entity. The determining module 802 is further configured to: calculate the distance between each vertex in each subgraph and the observation point corresponding to the first viewpoint image; determine the first subgraph in the multiple subgraphs based on the distance; the labeling module 803 is specifically configured to: label the geometric features formed by the first geometric entity belonging to the first subgraph with first category information.

[0141] Optionally, the first subgraph is the subgraph corresponding to the vertex with the smallest distance from the observation point among the vertices of the above subgraphs.

[0142] Optionally, each subgraph includes vertices and edges in the corresponding first geometric entity. The determining module 802 is further configured to: calculate the weight of each vertex in each subgraph of the multiple subgraphs, where the weight of each vertex is used to represent the importance of each vertex in the corresponding subgraph; determine the target vertex of the multiple subgraphs based on the weight of each vertex in the multiple subgraphs, where the target vertex of any subgraph is the vertex with the highest importance in any subgraph; calculate the distance between the target vertex of the multiple subgraphs and the observation point corresponding to the first viewpoint image; and determine the first subgraph of the multiple subgraphs based on the distance. The labeling module 803 is specifically configured to: label the geometric features formed by the first geometric entity belonging to the first subgraph with first category information.

[0143] Optionally, the first subgraph is the subgraph corresponding to the target vertex with the smallest distance from the observation point among the target vertices of the above subgraphs.

[0144] Optionally, the above three-dimensional model is a computer-aided design model.

[0145] Optionally, the above three-dimensional model is in the product data exchange standard format.

[0146] It should be noted that the acquisition module 801, determination module 802, and annotation module 803 described above can be implemented in software, hardware, or a combination of both. For example, the implementation of the determination module 802 will be described below. Similarly, the implementation of the other modules can be referenced to the implementation of the determination module 802.

[0147] As an example of a software functional unit, module 802 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, module 802 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0148] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0149] As an example of a hardware functional unit, module 802 may include at least one computing device, such as a server. Alternatively, module 802 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0150] The multiple computing devices included in module 802 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in module 802 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in module 802 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0151] In other embodiments, the acquisition module 801, determination module 802, and annotation module 803 described above can all be used to perform... Figure 1 Any step in the annotation method, the steps that each of these modules is responsible for implementing can be specified as needed, and implemented through them respectively. Figure 1 The different steps in the annotation method are used to annotate all the functions of device 800.

[0152] It should also be noted that, Figure 8The labeling device 800 is only illustrated by the above-mentioned division of functional modules. The names and numbers of these functional modules do not constitute a specific limitation. In actual applications, the labeling device 800 may include more or fewer modules. The internal structure of the device may be divided into other different functional modules as needed to complete all or part of the functions described above. For example, the above-mentioned functional modules may be divided into more modules, or the above-mentioned functional modules may be appropriately merged.

[0153] For example, Figure 9 This is a schematic diagram of another annotation device 900 provided in this application embodiment. The annotation device 900 performs the same function as the annotation device 800, and both can be used to implement... Figure 1 The difference between the two annotation methods lies in the way functional modules are divided. The annotation device 900 can be deployed on a cloud or local operating system; this application does not specifically limit its deployment.

[0154] The annotation device 900 includes a preprocessing component 910, a multi-view segmentation component 920, and an annotation component 930.

[0155] The preprocessing component 910 includes functional modules such as a data selector 911, a parser 912, a virtual camera 913, and a multi-view renderer 914. The data selector 911 preprocesses the 3D model data acquired by the annotation device 900 to filter information related to geometric entities, and can also filter dimensional parameters and other physical property information (such as density, material, color, etc.). The parser 912 parses the information filtered by the data selector 911 for processing by other modules. The 3D model data can be in STEP format; please refer to the previous description for details, which will not be repeated here. The virtual camera 913 is used to set the position, intensity, and type of the light source in the virtual environment where the 3D model is located. The multi-view renderer 914 captures 2D images of the 3D model from multiple perspectives within the aforementioned virtual environment, obtaining multiple view images; please refer to the previous description for details, which will not be repeated here.

[0156] The multi-view segmentation component 920 includes functional modules such as an image encoder 921, a cue word encoder 922, a mask decoder 923, and a prediction processor 924. The image encoder 921 converts the images from each viewpoint obtained by the multi-view renderer 914 into corresponding image embeddings. The cue word encoder 922 converts category information (such as user-inputted text cue words) into corresponding text embeddings. Each category corresponds to a type of processing feature that needs to be identified and labeled. There can be one or more categories. For details on category information and processing feature types, please refer to the previous description; further details are omitted here. The mask decoder 923 decodes the image and text embeddings to generate bounding boxes and corresponding segmentation masks. The prediction processor 924 predicts the category information to which the bounding boxes and corresponding segmentation masks belong.

[0157] The annotation component 930 includes functional modules such as a ray caster 931, a feature matcher 932, an AAG constructor 933, a topology traverser 934, and a label generator 935. The ray caster 931 projects the segmentation masks of the bounding boxes obtained from the images at various viewpoints onto the 3D model to determine the corresponding geometric entities. The feature matcher 932 calculates the centroids of each geometric entity projected by the ray caster 931 to filter out geometric entities belonging to interfering geometric features. The AAG constructor 933 constructs an attribute adjacency graph based on the remaining geometric entities after filtering by the feature matcher 932. The attribute adjacency graph includes multiple subgraphs, as detailed above. The topology traverser 934 determines the first subgraph from the multiple subgraphs; how the first subgraph is determined is also explained above. The label generator 935 is used to output the first geometric entity corresponding to the first subgraph determined by the topology traverser 934 (the first geometric entity can be referred to in the previous related introduction), and to label the geometric features formed by the first geometric entity corresponding to the first subgraph with the corresponding category information (labeling result).

[0158] It should be noted that, Figure 9 The division of the various functional modules in this document is for illustrative purposes only and does not constitute a limitation. In actual application scenarios, some modules may be combined or split.

[0159] Please see Figure 10 This application also provides a computing device 1000, including a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate with each other via the bus 1002. The computing device 1000 can be a server, laptop, tablet, desktop computer, edge device, smartphone, smart screen, etc., and this application does not specifically limit it, nor does it limit the number of processors and memories in the computing device 1000.

[0160] Bus 1002 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 10 The bus 1002 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1002 may include a path for transmitting information between various components of the computing device 1000 (e.g., memory 1006, processor 1004, communication interface 1008).

[0161] The processor 1004 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0162] The memory 1006 may include volatile memory, such as random access memory (RAM). The processor 1004 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0163] The memory 1006 stores executable program code. The processor 1004 executes the executable program code to implement... Figure 8 The labeling device 800 enables the functions of each module in the labeling device to achieve the purpose of this application. Figure 1 The operational steps in the method.

[0164] The communication interface 1008 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1000 and other devices or communication networks.

[0165] As one possible implementation, the computing device 1000 may also include a chip system, which includes a processor and a power supply circuit. The power supply circuit provides power to the processor, and the processor performs execution... Figure 1 The operational steps in the method are omitted here for brevity. The processor can be implemented using a CPU, or it can be implemented using a GPU, Neural Network Processing Unit (NPU), System on Chip (SoC), offloading card, accelerator card, or other processor or AI chip.

[0166] As one possible implementation, the computing device 1000 may include multiple types of processors 1004, that is, the computing device 1000 is a heterogeneous device. For example, the computing device 1000 includes a CPU and a GPU, and can be executed by at least one of the processors 1004. Figure 1 The operational steps in the method are omitted here for brevity.

[0167] like Figure 11 As shown, this application also provides a computing device cluster, which includes at least one computing device 1000. The memory 1006 of one or more computing devices 1000 in the computing device cluster may store the same memory for implementing… Figure 1 The instructions for the method.

[0168] In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store information for implementing the aforementioned... Figure 1 Part of the instructions of the method. In other words, a combination of one or more computing devices 1000 can jointly (in conjunction) execute instructions for implementing... Figure 1 The instructions for the method.

[0169] It should be noted that the memory 1006 in different computing devices 1000 within the computing device cluster can store different instructions, each for execution. Figure 8 The annotation device 800 performs some of the functions. That is, the instructions stored in the memory 1006 of different computing devices 1000 can realize the functions of one or more of the modules, namely the acquisition module 801, the determination module 802, and the annotation module 803.

[0170] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 12 One possible implementation is shown. For example... Figure 12As shown, two computing devices 1000A and 1000B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 1006 in computing device 1000A stores instructions for executing the functions of the acquisition module 801 and the determination module 802. Simultaneously, the memory 1006 in computing device 1000B stores instructions for executing the function of the annotation module 803.

[0171] It should be understood that Figure 12 The functions of computing device 1000A shown can also be performed by multiple computing devices 1000. Similarly, the functions of computing device 1000B can also be performed by multiple computing devices 1000.

[0172] This application also provides another type of computing device cluster. The interconnection relationships between the computing devices in this computing device cluster can be similarly referenced. Figure 12 The connection method of the computing device cluster. The difference is that the memory 1006 in one or more computing devices 1000 within this computing device cluster can store the same information used to implement the aforementioned... Figure 1 The instructions for the method.

[0173] In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store information for implementing... Figure 1 The instructions of the method. In other words, a combination of one or more computing devices 1000 can jointly execute instructions for implementing... Figure 1 The instructions for the method.

[0174] This application also provides a chip system, including a processor and a power supply circuit, wherein the power supply circuit supplies power to the processor, and the processor is used to execute... Figure 1 The operational steps of the method.

[0175] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that, when executed on a cluster of computing devices (including at least one computing device), cause the cluster of computing devices to perform... Figure 1 The operational steps in the method.

[0176] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on a cluster of computing devices (including at least one computing device), it causes the cluster of computing devices to execute... Figure 1 The operational steps in the method.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application. The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. In addition, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. The terminology used in the embodiments of this application is only for the purpose of describing specific embodiments, and is not intended to limit this application. The singular forms "a," "described," and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

Claims

1. A labeling method, characterized in that, The method includes: Obtain a first-view image of the 3D model, wherein the first-view image is a 2D image of the 3D model from the first viewpoint. Obtain information for the first category; First location information is determined based on the first viewpoint image and the first category information. The area indicated by the first location information in the first viewpoint image includes a first region, and the first region is the region corresponding to the first category information. Determine the first geometric entity in the three-dimensional model corresponding to the first location information; The first category information is labeled on the geometric features formed by the first geometric entity in the three-dimensional model.

2. The method according to claim 1, characterized in that, The first location information includes a bounding box and a segmentation mask.

3. The method according to claim 1 or 2, characterized in that, The step of determining the first geometric entity corresponding to the first location information in the three-dimensional model includes: Determine the multiple geometric entities corresponding to the first location information in the three-dimensional model; Determine the centroid of each of the plurality of geometric entities; A first geometric entity is determined from the plurality of geometric entities based on the centroid of each of the plurality of geometric entities, wherein the point corresponding to the centroid of the first geometric entity in the first view image is located within the area indicated by the first position information in the first view image.

4. The method according to any one of claims 1 to 3, characterized in that, The number of the first geometric entities is multiple, and the method further includes: An attribute adjacency graph is constructed for the plurality of first geometric entities, wherein the attribute adjacency graph is used to represent the topological structure of the plurality of first geometric entities, the attribute adjacency graph includes a plurality of subgraphs, each subgraph corresponds to at least one of the plurality of first geometric entities, different subgraphs correspond to different first geometric entities, and different subgraphs are divided into disjoint sets by the connected components in the attribute adjacency graph, and each subgraph is used to represent the topological structure of the corresponding first geometric entity.

5. The method according to claim 4, characterized in that, Each subgraph includes vertices and edges in the corresponding first geometric entity, and the annotation of the first category information on the geometric features formed by the first geometric entities in the 3D model includes: Calculate the distance between each vertex in each of the multiple subgraphs and the observation point corresponding to the first viewpoint image; Based on the distance, a first subgraph is determined among the plurality of subgraphs, and the geometric features formed by the first geometric entities belonging to the first subgraph are labeled with the first category information.

6. The method according to claim 5, characterized in that, The first subgraph is the subgraph corresponding to the vertex with the smallest distance from the observation point among the vertices of the plurality of subgraphs.

7. The method according to claim 4, characterized in that, Each subgraph includes vertices and edges in the corresponding first geometric entity, and the annotation of the first category information on the geometric features formed by the first geometric entities in the 3D model includes: Calculate the weight of each vertex in each of the plurality of subgraphs, whereby the weight of each vertex represents the importance of each vertex in the corresponding subgraph; The target vertex of each of the multiple subgraphs is determined based on the weight of each vertex in the multiple subgraphs, and the target vertex of any subgraph is the vertex with the highest importance in any given subgraph. Calculate the distance between the target vertices of the plurality of sub-graphs and the observation points corresponding to the first viewpoint image; Based on the distance, a first subgraph is determined among the plurality of subgraphs, and the geometric features formed by the first geometric entities belonging to the first subgraph are labeled with the first category information.

8. The method according to claim 5, characterized in that, The first subgraph is the subgraph corresponding to the target vertex with the smallest distance from the observation point among the target vertices of the plurality of subgraphs.

9. The method according to any one of claims 1 to 8, characterized in that, The three-dimensional model is a computer-aided design model.

10. The method according to any one of claims 1 to 9, characterized in that, The format of the three-dimensional model is the product data exchange standard format.

11. A labeling device, characterized in that, include: The acquisition module is used to acquire a first-view image of the 3D model, wherein the first-view image is a 2D image of the 3D model from the first viewpoint. The acquisition module is also used to acquire first category information; The determining module is configured to determine first location information based on the first viewpoint image and the first category information, wherein the area indicated by the first location information in the first viewpoint image includes a first region, and the first region is the region corresponding to the first category information. The determining module is further configured to determine the first geometric entity corresponding to the first position information in the three-dimensional model; The annotation module is used to annotate the first category information on the geometric features formed by the first geometric entity in the three-dimensional model.

12. The apparatus according to claim 11, characterized in that, The first location information includes a bounding box and a segmentation mask.

13. The apparatus according to claim 11 or 12, characterized in that, The determining module is specifically used for: Determine the multiple geometric entities corresponding to the first location information in the three-dimensional model; Determine the centroid of each of the plurality of geometric entities; A first geometric entity is determined from the plurality of geometric entities based on the centroid of each of the plurality of geometric entities, wherein the point corresponding to the centroid of the first geometric entity in the first view image is located within the area indicated by the first position information in the first view image.

14. The apparatus according to any one of claims 11 to 13, characterized in that, The number of the first geometric entities is multiple, and the determining module is further configured to: An attribute adjacency graph is constructed for the plurality of first geometric entities, wherein the attribute adjacency graph is used to represent the topological structure of the plurality of first geometric entities, the attribute adjacency graph includes a plurality of subgraphs, each subgraph corresponds to at least one of the plurality of first geometric entities, different subgraphs correspond to different first geometric entities, and different subgraphs are divided into disjoint sets by the connected components in the attribute adjacency graph, and each subgraph is used to represent the topological structure of the corresponding first geometric entity.

15. The apparatus according to claim 14, characterized in that, Each subgraph includes the vertices and edges of the corresponding first geometric entity. The determining module is further configured to: calculate the distance between each vertex in each of the plurality of subgraphs and the observation point corresponding to the first viewpoint image; and determine the first subgraph in the plurality of subgraphs based on the distance; The annotation module is specifically used to: label the geometric features formed by the first geometric entities belonging to the first subgraph with the first category information.

16. The apparatus according to claim 15, characterized in that, The first subgraph is the subgraph corresponding to the vertex with the smallest distance from the observation point among the vertices of the plurality of subgraphs.

17. The apparatus according to claim 14, characterized in that, Each subgraph includes vertices and edges in the corresponding first geometric entity, and the determining module is further configured to: Calculate the weight of each vertex in each of the plurality of subgraphs, whereby the weight of each vertex represents the importance of each vertex in the corresponding subgraph; The target vertex of each of the multiple subgraphs is determined based on the weight of each vertex in the multiple subgraphs, and the target vertex of any subgraph is the vertex with the highest importance in any given subgraph. Calculate the distance between the target vertices of the plurality of sub-graphs and the observation points corresponding to the first viewpoint image; The first subgraph among the plurality of subgraphs is determined based on the distance; The annotation module is specifically used to: label the geometric features formed by the first geometric entities belonging to the first subgraph with the first category information.

18. The apparatus according to claim 15, characterized in that, The first subgraph is the subgraph corresponding to the target vertex with the smallest distance from the observation point among the target vertices of the plurality of subgraphs.

19. The apparatus according to any one of claims 11 to 18, characterized in that, The three-dimensional model is a computer-aided design model.

20. The apparatus according to any one of claims 11 to 19, characterized in that, The format of the three-dimensional model is the product data exchange standard format.

21. A computing device cluster, characterized in that, The system includes at least one computing device, each computing device including a processor and a memory, wherein the memory of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 10.

22. A computer program product, characterized in that, Includes instructions that, when executed by a computing device cluster, cause the computing device cluster to perform the method as described in any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that, Includes instructions that, when executed by a computing device cluster, cause the computing device cluster to perform the method as described in any one of claims 1 to 10.

Images