Annotation Device

Abstract

To provide an annotation apparatus and a method that enables a simple and highly accurate annotation.SOLUTION: An annotation apparatus is configured using a calculator and annotates 3D shape data. The annotation apparatus includes an input unit that inputs a 3D CAD model, a text, and definition information of a label, an arithmetic unit that generates an annotation data set through annotation processing, a storage unit that stores the generated annotation data set, and an output unit that outputs a processing result of the arithmetic unit. The arithmetic unit includes an annotation unit that acquires annotation information including a partial shape of the 3D CAD model and a label according to a unique expression in the text or label definition information, and a metadata generation unit that generates metadata that associates the annotation information with the 3D CAD model and a partial shape of heterogeneous format data derived from the 3D CAD model. The annotation data set including the generated metadata, the 3D CAD model to which the metadata refers, and the text are stored in the storage unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention provides annotation To the device Regarding. [Background technology]

[0002] In product design, it is common to use 3D CAD (Computer Aided Design). 3D CAD (hereafter referred to as CAD) is a tool that creates the 3D shape of a product on a computer using techniques including solid modeling and parametric modeling, according to the designer's operations. In many CADs, the 3D shape is represented by BREP (Boundray REPresentation), which describes the shape's solids, faces, edges, points, and their topology information. Hereinafter, the 3D shape created by CAD will be referred to as a CAD model.

[0003] Patent Document 1 is known as a technology for annotating a CAD model or a technology that uses annotation information for a CAD model.

[0004] Patent Document 1 proposes a device that uses a data set of predefined shapes and labels to recognize labels of new shapes and searches for design rules related to the recognized labels when determining design rules, which are constraints on CAD models that stem from product requirements, manufacturability, etc., based on shapes. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2020-95378 Summary of the Invention [Problem to be solved by the invention]

[0006] However, Patent Document 1 has the following problem: Patent Document 1 presents a device that makes labeling of partial shapes more efficient by inputting a label to a base shape and then deforming it to create a new shape with the same labeling, but when labeling multiple different CAD models, this device does not make labeling more efficient, and it is necessary to repeatedly open the CAD model, manually select and label the partial shapes, and then close the CAD model, which results in a problem of a large amount of work.

[0007] Furthermore, in Patent Document 1, the format of the label item name and its contents is fixed or embedded in the device, and the format cannot be changed externally. Meanwhile, in general 3DA, the label format is not standardized. To accommodate various applications, it is necessary to perform uniform annotation on multiple CAD models according to label definition information tailored to the purpose. Examples of annotation applications include using a dataset of pairs of CAD model part shapes and annotation labels for machine learning or for setting thresholds for shape recognition. Therefore, a challenge is to perform uniform annotation using a label format tailored to the purpose.

[0008] Furthermore, Patent Document 1 only deals with annotations targeting labels. In product design, review texts are created for defects in CAD models during design reviews, and the texts refer to the partial shapes of the CAD models using demonstrative terms and illustrations. However, these are not associated with the 3D CAD data, and the referred partial shapes must be discovered manually. Therefore, the challenge is to associate expressions in the text with the partial shapes of the CAD models.

[0009] Furthermore, Patent Document 1 does not mention a method for saving input annotations. CAD models are often converted into different formats, such as intermediate files or mesh models, and 3DA information is lost during the conversion. For visualization purposes in production sites, CAD models are often converted into surface mesh models for use. Therefore, maintaining annotation information in CAD models converted into different formats, or identifying the partial shapes of the different formats using the partial shapes annotated in the CAD model as a key, is a challenge.

[0010] In view of the above, an object of the present invention is to provide an annotation device and method that enable simple and highly accurate annotation. [Means for solving the problem]

[0011] In light of the above, the present invention is an annotation device for annotating 3D shape data, comprising: an input unit for inputting a 3D CAD model, a sentence, and label definition information; a calculation unit for generating an annotation dataset by annotation processing; a memory unit for storing the generated annotation dataset; and an output unit for outputting the processing result of the calculation unit, wherein the calculation unit comprises: an annotation unit for obtaining annotation information including a partial shape of the 3D CAD model and a label according to the named entity in the sentence or label definition information; and a metadata generation unit for generating metadata that associates the annotation information with the 3D CAD model and the partial shape of heterogeneous format data derived from the 3D CAD model, wherein the generated metadata, the 3D CAD model referenced by the metadata, and the annotation dataset including the sentence are stored in the memory unit. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide an annotation device and method that enable simple and highly accurate annotation. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing an example of the configuration of an annotation device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an example of a screen 20 for setting and inputting a CAD model to be annotated, etc. [Figure 3] FIG. 3 is a diagram showing an example of an annotation screen 30 for a CAD model and label values. [Figure 4] FIG. 4 is a diagram showing an example of an annotation screen 40 for a CAD model and text. [Figure 5] FIG. 10 is a diagram showing an example of label definition data D2. [Figure 6] FIG. 10 is a diagram showing an example of a processing flow in the annotation metadata generation / association unit 17. [Figure 7] FIG. 10 is a diagram showing an example of metadata and annotations embedded in a shape model. [Figure 8] FIG. 10 is a diagram showing an example of an annotation process using a recognition function. [Figure 9] FIG. 10 is a diagram showing an example of a learning process flow of the shape / label / document feature recognition unit. [Figure 10] FIG. 10 is a diagram showing an example of an inference processing flow of a shape / label / document feature recognition unit. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] In the following description of the embodiments, annotation refers to the addition of relevant information (metadata) to certain data. Annotation (3DA) in 3D CAD refers to the addition of metadata (such as text, dimensions, tolerances, welding symbols, and surface finishes) to certain data (in this case, a CAD model) related to the shape of the CAD model. In 3DA, annotations are associated with the shape of the CAD model and added as annotations, which can be created, saved, and displayed in CAD software. 3DA information is stored along with CAD model information in part or assembly documents output by the CAD software. 3DA is primarily used to describe product requirements, manufacturing requirements, and manufacturing instructions, and is said to facilitate the automation of manufacturing procedures.

[0016] Attempts to annotate data are also increasing in other fields. In particular, annotations that define and label regions in images and text are useful as training data for image and language recognition using machine learning. These annotation data are mainly created using annotation tools, either manually or with the assistance of programs such as machine learning.

[0017] The annotations prepared as training data are characterized by being created in a unified annotation format and standard according to the purpose so that the same training process can be applied to a large amount of data. The annotated information is saved in a file format such as JSON (JavaScript Object Notation) by describing the annotation information and reference information to images and text. [Example]

[0018] 1 shows an example of the configuration of an annotation device according to an embodiment of the present invention. The annotation device 1 is configured using a computer and includes an input unit 10, an output unit 15, an operation unit 16, Annotation dataset storage unit 9, and a calculation unit 18. The processing functions of the calculation unit 18 include an annotation unit 14, an annotation metadata generation / association unit 17, and a shape / label / document feature recognition unit 19.

[0019] According to the annotation device 1 of Figure 1, a CAD model set D1 to be annotated, label definition information D2 defining the label content to be annotated, and a text document D3 to be associated with the CAD model are input, and an annotation screen is presented in accordance with these inputs. Annotation data D4 is created based on user operation, and the new annotation data D4 is stored.

[0020] Specifically, various data are first input from the input unit 10. A CAD model set D1 of multiple annotation targets is input to the CAD model set input unit 11 of the input unit 10. Label definition information D2 of the labels to be annotated is input to the label definition information input unit 12. The label definition information D2 includes the identification name of the label definition, the item names of multiple labels, the type of label value, the values and ranges allowed for the label value, the type of geometric entity associated with the label (face, edge, solid, etc.), and the type of named entity of the sentence to be annotated. The document input unit 13 inputs text information (document D3) of the sentence to be associated with the CAD model. Hereinafter, a named entity refers to a meaningful character string extracted from a portion of a sentence.

[0021] Next, we will explain the various processing functions of the calculation unit 18. First, the annotation unit 14 annotates labels for partial shapes in the CAD models according to the labels defined by the label definition information D2 inputted in the label definition information input unit 12 for multiple CAD model sets D1 inputted in the CAD model set input unit 11 and multiple sentences (documents D3) inputted in the document input unit 13 as needed.

[0022] Here, a label is an association of a value of a type defined in the label definition information D2 with each label item defined in the label definition information D2. When annotating a portion of a sentence to a CAD partial shape, the reference relationship between the named entity in the classified sentence and the CAD partial shape is annotated as a label in accordance with the label definition information D2. The annotation unit 14 can generate a user interface that enables the task of creating the above-mentioned annotations and interactively create annotations based on user operations.

[0023] The annotation metadata generation / association unit 17 in the calculation unit 18 compiles the annotation data, including the labels, CAD models, and text created by the annotation unit 14, into annotation metadata D4, which includes label information and reference information to the CAD models and text. At the same time, information such as identifiers for associating the annotation metadata D4 with partial shapes is embedded in the form of a 3D annotation or mesh group into heterogeneous data, such as the CAD model to be annotated or a mesh model derived from the CAD model.

[0024] The annotation dataset storage unit 9 stores the annotation metadata D4 generated by the annotation metadata generation / association unit 17, the CAD model, and other data. The annotation metadata D4, CAD model, and the heterogeneous data and text derived therefrom are collectively referred to as the annotation dataset. Storage formats include files, document-based databases, and object storage. At the same time, the stored annotation dataset can be read out for use by the recognition unit 19 (described later) or externally. Methods for acquiring the annotation dataset include database queries and communications such as HTTP (HyperText Transfer Protocol).

[0025] In many cases, annotation data is stored together with a 3D CAD model as its additional information, but in the first embodiment of the present invention, the annotation data is stored in association with the annotation data set storage unit 9 on the annotation device side, rather than on the 3D CAD side. This simplifies the handling of annotations when associating multiple CAD models.

[0026] The shape / label / document feature recognition unit 19 in the calculation unit 18 uses the information in the annotation dataset stored in the annotation dataset storage unit 9 to search for similar shapes, recognize named entities in sentences, and predict labels from shapes, thereby assisting the user's annotation operations in the annotation unit 14, for example by automatically predicting and recommending annotation labels for a CAD model shape selected by the user.

[0027] The output unit 15 is an information processing device that presents the above-mentioned user interface to the user. The operation unit 16 is an input device such as a mouse, touch panel, or keyboard that is used when the user creates an annotation using the above-mentioned user interface.

[0028] When implementing each function of the CAD model set input unit 11, label definition information input unit 12, document input unit 13, annotation unit 14, annotation metadata generation / association unit 17, shape / label / document feature recognition unit 19, and annotation dataset storage unit 9 in the annotation device 1 as a computer system, it is possible to use a configuration consisting of a single information processing device and the software stored therein, or a configuration consisting of a network connecting a server running software located remotely from the user with the output unit 15 and operation unit 16 in the user operation terminal, or a configuration that partially combines both of these.

[0029] Next, an example of the screen configuration output to the output unit 15 will be described with reference to Figures 2 to 4. First, Figure 2 shows an example of a screen 20 for setting and inputting a CAD model or the like to be annotated.

[0030] The setting and input screen 20 is divided into four small screens, and the small screen 21 displays the dataset name 207 and the save destination 208, as well as a button B1 that gives an instruction to start annotation when the input settings are complete.

[0031] Small screens 22, 22, and 23 are setting and input screens for CAD model set D1, label definition information D2, and document D3, respectively. When a data set name 207 is specified, the corresponding information D1, D2, and D3 are displayed, and by operating buttons B2, B3, and B4, the information to be annotated is appropriately selected or added from this, and then imported into the computer for processing.

[0032] On the small screen 22, a CAD model list D1 to be annotated is added by the user operating a button B2.

[0033] On the small screen 23, the user adds label definition information D2 for annotations to the CAD model described above by operating button B3. The label definition information D2 used for annotations can be enabled or disabled by operating a checkbox. An example of label definition information will be described later in Figure 5. Methods for adding label definition information D2 include using a user interface form (no example screen) and inputting it using a structured data format such as JSON.

[0034] On the small screen 24, a document file is added, which is text information to be input to the document input section 13. The document file includes text data and more complex rich text data.

[0035] On the input screen 20 of the overall configuration in Figure 2, after setting the name / identifier name 207 of the dataset and the location 208 where the dataset will be saved on the small screen 21, pressing the start annotation button B1 will transition to the annotation screens 30 and 40 shown in Figures 3 and 4 where annotation will be performed, and the input CAD model D1, label definition information D2, and text D3 will be stored in the annotation dataset storage unit 9.

[0036] For example, the display example in Figure 2 shows an example in which multiple CAD models D1 are input and are set to be annotated with labels according to label definition information D2 entitled "Rib base annotation" and "Rib boss annotation."

[0037] 3 shows an example screen configuration 30 for annotating labels based on label definition information D2 for multiple CAD models D1 in the annotation unit 14. This screen 30 also has a multiple small area screen configuration. Small areas 31 and 32 on the left are screens for selecting the CAD model D1 and label definition information D2 to be annotated, small areas 34 and 35 on the right are screens for setting processing conditions for the label definition information D2, and small area 33 in the center is a CAD screen.

[0038] In the left section 31 of this example screen, the CAD models D1 set in the CAD model set input section 11 are displayed in a list, and can be selected from the list by the user operating button B5. The selected CAD model can also be changed by operating button B5. In the label definition display area 32, a list of label definitions that are the annotation targets is displayed. Here, annotations for ribs, bosses, and rib bases are displayed in a list, and it is clearly indicated by bold text that the rib base annotation is the currently targeted operation.

[0039] In the right part 35 of this screen example, when the add button B6 or B7 is pressed, annotation based on the label definition D2 selected by this button B6 or B7 is started for the currently selected CAD model D1.

[0040] The CAD model display section 33 in the center of this screen example displays a 3D model of the CAD model D1 selected on the CAD model list screen 31. In the CAD model display section 33, partial shapes (solids, faces, edges, points, or a combination of multiple of these) in the CAD model can be selected by operating the operation section 16 in Figure 1.

[0041] The example screen in FIG. 3 shows an example in which multiple faces corresponding to rib roots are selected in the CAD model display area 33 after selecting "Add rib root" with the annotation addition button B6.

[0042] In the labeling unit 34 of this screen example, label item values are entered for the selected CAD partial shape based on the label definition information. The input area for the label item value is realized in the form of a text box 308, a drop-down menu option 309, or the like, based on the information on the type of label item value in the label definition information. When the save button 310 is pressed, annotation information is generated in the annotation metadata generation / association unit based on the information on the selected CAD model, its partial shape, and the entered label item value, and the information is saved in the annotation dataset storage unit.

[0043] 4 shows a screen 40 on which annotation unit 14 performs annotation for multiple CAD models and multiple sentences, associating CAD model partial shapes with named entities in the sentences. This screen 40 is also configured as a multiple-subarea screen. Subareas 41 and 42 on the left are screens for selecting CAD model D1 and sentence document D3 to be annotated, subarea 43 on the right is a screen for setting processing conditions for sentence document D3, and subarea 44 in the center is a CAD screen. Subarea 41 on the left is the same as CAD model list D1 area 31 in FIG. 3 described above.

[0044] The text selection section in the small area 42 on the left displays the name of the label definition information for the annotation that targets the text, which was set in the label definition information input section 12, and the text data entered in the document input section 13. In the text list, text data enclosed in a rectangular area is selected, and the selected text data can be switched by clicking the text name or button B8.

[0045] The sentence selected in the sentence selection section 42 is displayed in the sentence display section of the small area 43 on the right side. A named entity 406 in the sentence can be selected by operating the operation section 16 in Figure 1 on the sentence displayed here. In Figure 4, the expressions "base of the rib" and "small corner R" have been selected by a selection operation. In the named entity label section 407, a label for the named entity 406 defined in the label definition information D2 is set in 407 for the named entity 406 selected and focused in the sentence display section 43.

[0046] In this example, the label "Part Name" is selected from the drop-down list for the named entity 406 "Rib base." In the CAD display section 44, the CAD model selected in the CAD model list D1 is displayed in the small area 41. With the named entity 406 selected in the sentence display section 43, by selecting a partial shape 408 of the CAD model in the CAD display section 44, an annotation can be created that associates the selected partial shape 408 with the selected named entity 406. The user creates annotations by switching between the CAD model list D1 and the sentence D3. It is also possible to create annotations that associate multiple CAD models with one sentence.

[0047] FIG. 5 shows an example of a label definition D2 input in the label definition information input unit 12. Here, the label definition is expressed using JSON. Label example 501 is an example of a label definition named "rib / boss annotation." The label definition name is input in the name field, and the shape type of the annotation target (surface in this example) is input in the target field. The label_definition field is input with key-value format data, with the label item name as the key and the label value type as the value. In this example, for example, the label item "shape type" is defined as an option with classes such as rib and boss, the label item "shape name" is defined as a string, and the label item "dimensions" is defined as a float with a value between 0 and 10.

[0048] Label definition example 502 is an example of a label definition with the name "Rib base annotation." The difference from the previous example 501 is that the field value of the shape (target) to be annotated is set to edges or faces (edges|surface). The shape type selected in the target (field) becomes selectable in the CAD model display area 33 in Figure 3.

[0049] Label definition example 503 shows an example of a label definition with the name "text annotation." In this example, a character string (text) is set in the target field, and in this case, a screen for annotating text and CAD models (401 in Figure 4) is displayed as the annotation screen. The Entities field stores a list of classification class labels for named entities in text. In this example, named entities in text are classified into two types: "body part name" and "defect details."

[0050] FIG. 6 shows a flowchart of the processing in the annotation metadata generation / association unit 17. First, the processing steps S61 Next, a plurality of CAD models D1, label definition information D2, and documents D3 are obtained from the input unit 10. Next, processing steps S62Then, the named entities in the labels or sentences of the annotations created by the annotation unit 14 and the partial shapes of the CAD model are obtained.

[0051] Next processing step S63 In this system, a unique identifier is issued for the CAD model part shape included in the annotation. This identifier may be an ID issued by the CAD software that created the CAD model, an ID of a shape included in the CAD model, or a unique character string generated within this system.

[0052] Next processing step S64 Next, annotation metadata D4 is created, which is data that associates the label values created by the annotation unit 14 with the identifiers of the corresponding CAD model part shapes. The annotation metadata D4 is created in text or binary format, for example, in a data format that can express dictionary-type or list-type data structures such as JSON.

[0053] Next processing step S65 In this example, a 3D annotation containing identifier information is added to the CAD model D1 for the partial shapes in the CAD model for which an identifier has been issued. At this time, the 3D annotation to be added may include not only the identifier information but also the location where the annotation data set was saved and information about the labels created by the annotation unit 14. Furthermore, the CAD model is converted to create heterogeneous data formats, such as BREP, surface mesh, volume mesh, and voxel formats, and the identifiers of the CAD model partial shapes described above are written to the partial shapes, such as surfaces and bodies, in the heterogeneous data formats. At this time, label information other than identifiers may also be embedded in the partial shapes of the heterogeneous data formats.

[0054] In the above process, by adding annotation data directly to the CAD model or its converted heterogeneous data, it becomes possible to refer to the annotation metadata from the CAD model data and access and use the annotation information stored outside the model.In addition, by using the annotation metadata from the 3D annotation information of the CAD model, it becomes possible to know where the part shape of the annotation target corresponds in the heterogeneous format.

[0055] Finally, the processing step S66 Now, the 3D annotated CAD model, heterogeneous format data, and annotation metadata are saved in the save location specified in 208 of FIG.

[0056] 7 shows examples of annotation metadata 701, 3D annotations 702 to a CAD model, and annotations 703 to heterogeneous format data generated by converting the CAD model. The representation here uses JSON.

[0057] In the annotation metadata 701 of Figure 7, the annotation dataset is represented in a structured data format such as JSON, and at least the following information is stored: the name of the dataset (dataset), shape models (models), text (text), label definitions (label_definitions), and annotation data (annotations).

[0058] The shape model (models) field stores multiple pieces of information, including the path to the save destination for the CAD model and the converted heterogeneous format data, and the shape model ID. In this example, the model with ID 1 is associated with a CAD model with the file path xxxx.step and a mesh model with the file path xxx.stl.

[0059] The texts section stores a list of references to texts used for annotation. In this example, two texts with file paths "design_review1.txt" and "design_review2.txt" are specified as annotation targets.

[0060] In the label definition (label_definitions), the label definition information shown in Fig. 5 is stored as a list. In this example, the same data as 501 in Fig. 5 is stored at the beginning of the list.

[0061] The annotation data (annotations) stores annotation information created by the annotation unit 14 in a dictionary format or the like. In this example, data corresponding to three label definitions is stored: a rib root annotation, a rib / boss annotation, and a text annotation. In this example, the first element in the rib root annotation list is an annotation with a CAD model ID of 1 (model) and a label (label) indicating an angle R of 5 and a rib type of triangle for the face (shape) in the CAD model with an identifier of face:0001. Also, the first annotation element in the text annotation list is an annotation that sets the label "part name" (entity) to the 40th-50th characters of the text (text) design_review1.txt, and associates this with a face (shape) with a CAD model ID of 1 and a partial shape identifier of "face:001."

[0062] 3D annotation 702 written in a CAD model in FIG. 7 shows an example in which the identifier face:0001 is 3D annotated and saved together with related label information for the surface annotated in the annotation section for the CAD model xxxx.step.

[0063] Figure 7 shows an example of a heterogeneous format file 703 obtained by converting a CAD model, along with an example of annotations for the mesh file. This example shows a portion of the mesh file, with the line beginning with "f" representing a face by combining three predefined vertex numbers and normal numbers. The line beginning with "annotation_group" indicates that the face between the following occurrences of "annotation_group" corresponds to "face:0001."

[0064] Fig. 8 shows an example of assisting annotation work in the annotation unit 14 using the shape / label / document feature recognition unit 19. Fig. 8 shows an example of the screen configuration at this time. This screen is basically the same as the example screen configuration 30 for annotating labels based on the label definition information D2 in Fig. 3, but differs in that a shape recognition button B8 has been added to the small area 35, and prediction results are displayed in the text box 308 of the small area 34 and the options 309 of the drop-down menu.

[0065] The screen in Figure 8 is used and displayed as follows. First, in this example, it is assumed that the user is annotating the rib base. The user first selects the partial shape 306 they wish to annotate in the small area 33 by clicking or touching it. At this time, information about the selected partial shape 306 is sent to the recognition unit 19, and the recognition unit 19 receives a predicted value for the label of the selected partial shape 306.

[0066] The received label prediction values are automatically entered into the label input section 308 of the small area 34 and displayed as predicted result values. These prediction results can be adopted as is or modified by the user, and annotations can be added by operating the add button B6 at the base of the rib.

[0067] Furthermore, when the shape recognition button B8 is pressed, the entire CAD model and the currently created label definition are sent to the recognition unit 19, which then recognizes the shape and highlights a portion 804 in the CAD model that is predicted to correspond to the rib root in the small area 33. Furthermore, if the user selects the highlighted portion 804, the shape of the selected portion can be edited. Similarly, the predicted value of the label can be displayed for the highlighted portion 804 (in the text box 308 in the small area 34), allowing the user to modify it as necessary and register the label.

[0068] 9 and 10 show flowcharts of the processing of the shape / label / document feature recognition unit 19. The processing of the shape / label / document feature recognition unit 19 includes a learning process starting from S901 shown in Fig. 9 and an inference process starting from S904 shown in Fig. 10.

[0069] 9 is executed as a batch process or manually when a dataset is created. In the learning process, the annotation dataset D4 stored in the storage unit 18 is read in the first processing step S901. This processing may be performed independently for each individual dataset, or preprocessing such as merging datasets with the same label definition or selecting annotation data based on certain criteria may be performed.

[0070] Next, in processing step S902, learning is performed using the acquired data set D4. Learning is performed from several perspectives. One of these is learning the relationship between label definitions and partial shapes, in which learning is performed to predict partial shapes to which labels should be assigned based on label definition information. This is done by setting up a rule-based program based on geometric calculations in advance and optimizing parameters such as thresholds for geometric calculations to minimize prediction errors in the data set. Alternatively, machine learning is used to classify and predict the surfaces and lines of CAD shapes using graph kernel learning, graph neural networks, or deep learning using mesh data converted from CAD shapes, implicit functions, graph neural networks for voxel data, convolution, and Transformer.

[0071] The second learning perspective is learning the relationship between subshapes and labels, where the selected subshape is used as input and the corresponding label is output. For this prediction, the surrounding shapes of the selected subshape or the entire shape of the CAD model can also be used as input as supplementary information. The analysis method can be the same as the learning method described above, and a model is trained to predict the label value for each subshape for each label item.

[0072] The third learning perspective is learning named entities in text. Here, methods using corpora or dependency relationships, rule-based methods, and predictions using neural networks can be used, and learning is carried out so that named entities in text specified by label definitions can be extracted and their labels can be predicted.

[0073] The learning process in process step S902 consists of these three learning tasks, which may be learned independently, or may be learned simultaneously or end-to-end using multitask or transfer learning.

[0074] Once the above learning model has been trained, in processing step S903, the information on dataset D4, the label definition of that dataset, and the above learning model are saved as a set.

[0075] 10 , in processing step S904, a learning model corresponding to the label definition of the annotation is obtained in accordance with a request from the annotation unit 14. Thereafter, if the annotation unit 14 requests prediction of a partial shape to be labeled, a prediction is made using a learning model of the relationship between the label definition and the partial shape in processing step S905. If the annotation unit 14 requests prediction of a label value from the partial shape and label definition information, a prediction is made using a learning model of the relationship between the partial shape and the label in processing step S906. If the annotation unit 14 requests recognition of named entities in a sentence, a prediction is made using a learning model for recognizing named entities in a sentence in processing step S907. In processing step S908, the prediction results of these learning models are sent to the annotation unit 14.

[0076] According to the above-described embodiment of the present invention, by defining and inputting label definition information in advance, annotations in the same format can be applied to multiple CAD models, and when using the annotation information as a data set for machine learning or other programs, there are no format discrepancies or fluctuations in label definitions, making it possible to process annotation information in bulk. At the same time, compared to manual annotation, input forms and input values can be validated in accordance with the label definition information, reducing annotation labor and input errors.

[0077] Furthermore, according to the embodiment of the present invention, by inputting multiple CAD models simultaneously, the work of opening and closing CAD models when annotating the partial shapes of the CAD models is eliminated, reducing the amount of work required for annotation. In addition to manual annotation, automatic annotations based on shape recognition, label recognition, etc. can be corrected by the user, reducing the amount of work required for labeling and shape selection.

[0078] Furthermore, according to an embodiment of the present invention, by performing annotations that associate text with CAD model partial shapes, the design review text and CAD model partial shapes are associated in the data, making it possible to reference text from the CAD model and partial shapes from text.

[0079] According to an embodiment of the present invention, a dataset is centrally stored in association with multiple CAD models, thereby facilitating scanning of annotation information and shape information of multiple CAD models in the dataset. Furthermore, when a CAD model is converted into a different format (different CAD format, surface mesh, voxel, etc.), the data before and after conversion are centrally managed, so that annotation information is not lost even after conversion and can be used. [Explanation of symbols]

[0080] 1: Annotation device 10: Input section 11: CAD model set input section 12: Label definition information input section 13: Document input section 14: Annotation section 15: Output section 16:Operation unit 17: Annotation metadata generation / association part 18: Arithmetic section 19: Shape / Label / Document Feature Recognition Unit 19 9: Annotation dataset storage

Claims

1. An annotation device that annotates 3D shape data, The system includes an input unit for inputting a 3D CAD model, a sentence, and definition information of a label, a calculation unit for generating an annotation data set by annotation processing, a storage unit for storing the generated annotation data set, and an output unit for outputting a processing result of the calculation unit, The calculation unit includes an annotation unit that obtains annotation information including a partial shape of the 3D CAD model and a label according to the definition information of the named entity in the sentence or the label, and a metadata generation unit that generates metadata that associates the partial shapes of the 3D CAD model and heterogeneous format data derived from the 3D CAD model with the annotation information, and the annotation device is characterized in that the generated metadata, the 3D CAD model referenced by the metadata, and an annotation dataset including the sentence are stored in the storage unit.

2. The annotation device according to claim 1, the calculation unit includes a recognition unit that learns the label definitions of the annotation dataset in the storage unit and the corresponding relationships between the geometric shapes and the label annotations, and stores the learned model; the recognition unit determines a predicted annotation label for the input partial shape in accordance with a request from the annotation unit; The annotation device is characterized in that the output unit outputs the annotation label determined by the recognition unit.

3. The annotation device according to claim 1, An annotation device characterized by comprising a recognition unit that extracts sentence data and named entity labels contained in label definition information from the annotation dataset in the memory unit, learns to recognize named entities in sentences, performs the learned named entity recognition on sentences read by the annotation unit in accordance with a request from the annotation unit, outputs the results to the output unit, and records them in the memory unit.

4. The annotation device according to claim 1, An annotation device characterized by having a recognition unit that learns the relationship between which partial shapes in a 3D CAD model are selected and the label definitions in the annotation data set in the memory unit, and recommends partial shapes to be annotated for the label definition information in accordance with a request from the annotation unit.

5. The annotation device according to claim 1, The annotation device is characterized in that the metadata generation unit embeds a common partial shape identifier into the metadata, the partial shapes of the 3D CAD model referenced by the generated metadata, and the partial shapes of the heterogeneous format data derived therefrom.

Images