Model training device, model training method, and program

Abstract

A model training device (10) comprises a first patch generation unit (110), a second patch generation unit (120), and a training unit (130). The first patch generation unit (110) uses three-dimensional data representing the shape of an object to generate at least one first patch, which is a subset of the three-dimensional data, and stores the at least one first patch into a first storage unit (20). The second patch generation unit (120) reads one first patch from the first storage unit (20), and generates at least one second patch, which is a subset of the first patch. The training unit (130) trains a model for evaluating a three-dimensional shape by using the second patch as training data. Then, the second patch generation unit (120) and the training unit (130) repeat the processing until a criterion is satisfied.

Description

[Technical Field]

[0001] The present invention relates to a model training device, a model training method, and a recording medium. [Background technology]

[0002] Detecting anomalies in objects using 3D data that indicates the shape of the object has been studied, and in recent years, detecting these anomalies using models trained by machine learning has been studied.

[0003] For example, Non-Patent Document 1 describes that a plurality of spherical patches, which are subsets of three-dimensional data, are generated from the three-dimensional data so as to cover the entire three-dimensional data, and these spherical patches are used as training data.

[0004] The technology described in Non-Patent Document 1 above cannot learn all of the spherical patches that can be generated. Therefore, the performance when inferring using the generated model tends to depend on the patch division during inference.

[0005] In contrast, Non-Patent Document 2 describes that in each epoch during training, 3D data is read, spherical patches are randomly generated, and the spherical patches are input to a point cloud learner. In Non-Patent Document 2, the spherical patches are also generated so as to cover the entire 3D data. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] "Transforms," in the API document of TorchPoints3D, [Retrieved March 4, 2022]<URL:https: / / torch-points3d.readthedocs.io / en / latest / src / api / transforms.html> [Non-patent document 2] Thomas, et al., "KPConv: Flexible and deformable convolution for point clouds," 2019, [Retrieved February 28, 2022]<URL:https: / / arxiv.org / pdf / 1904.08889.pdf> Summary of the Invention [Problem to be solved by the invention]

[0007] In the above-mentioned Non-Patent Document 2, it was necessary to read out three-dimensional data indicating the shape of the target object at each epoch during training, which increased the load on the device that performs the training.

[0008] In view of the above-mentioned problems, one example of the object of the present invention is to provide a model training device, a model training method, and a recording medium that impose a small load when training a model that evaluates three-dimensional data that indicates the shape of an object. [Means for solving the problem]

[0009] According to one aspect of the present invention, a first patch generation unit generates at least one first patch, which is a subset of three-dimensional data representing a shape of an object, using the three-dimensional data, and stores the first patch in a storage unit; second patch generating means for reading any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training means for training a model for evaluating a three-dimensional shape using the second patch as training data; Equipped with A model training apparatus is provided, in which the second patch generating means and the training means repeat the process until a criterion is met.

[0010] According to one aspect of the present invention, a computer generating at least one first patch that is a subset of the three-dimensional data representing the shape of the object, and storing the first patch in a storage means; reading out any of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training a model for evaluating a three-dimensional shape using the second patch as training data; A model training method is provided that repeats the generation of the second patch and the training of the model until a criterion is met.

[0011] According to one aspect of the present invention, a computer includes: a first patch generation function that uses three-dimensional data indicating the shape of an object to generate at least one first patch that is a subset of the three-dimensional data and stores the first patch in a storage means; a second patch generation function that reads out any one of the first patches from the storage means and generates at least one second patch that is a subset of the first patch; a training function for training a model for evaluating a three-dimensional shape using the second patch as training data; It records the program that allows you to A computer-readable medium is provided in which the second patch generation function and the training function repeat the process until a criterion is met. [Effects of the Invention]

[0012] According to one aspect of the present invention, it is possible to provide a model training device, a model training method, and a recording medium that impose a small load when training a model that evaluates three-dimensional data that indicates the shape of an object. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an overview of a model training device according to an embodiment. [Figure 2] FIG. 1 is a diagram showing the configuration of a model training device together with the environment in which the model training device is used. [Figure 3] FIG. 10 is a diagram for explaining the processing performed by a first patch generation unit of the model training device. [Figure 4] FIG. 10 is a diagram for explaining the processing performed by a second patch generation unit of the model training device. [Figure 5] 10A and 10B are diagrams for explaining an example of a problem that may arise when a first patch generating unit is used. [Figure 6] 10A and 10B are diagrams for explaining an example of a problem that may arise when a second patch generating unit is used. [Figure 7] 6A and 6B are diagrams for explaining a method for solving the problem shown in FIG. 6, and FIG. 6C is a diagram for explaining a method for solving the problem shown in FIG. 5. [Figure 8] FIG. 2 is a diagram illustrating an example of a hardware configuration of a model training device. [Figure 9] FIG. 10 is a diagram illustrating a first example of processing performed by a first patch generation unit of the model training device. [Figure 10] FIG. 10 is a diagram illustrating a second example of processing performed by the first patch generation unit of the model training device. [Figure 11] FIG. 10 is a diagram illustrating an example of processing performed by a second patch generation unit and a training unit of the model training device. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and their description will be omitted where appropriate.

[0015] 1 is a diagram showing an overview of a model training device 10 according to an embodiment. The model training device 10 includes a first patch generating unit 110, a second patch generating unit 120, and a training unit .

[0016] The first patch generation unit 110 uses three-dimensional data indicating the shape of an object to generate at least one first patch that is a subset of the three-dimensional data and stores the first patch in the first storage unit 20. The second patch generation unit 120 reads out one of the first patches from the first storage unit 20 and generates at least one second patch that is a subset of the first patch. The training unit 130 uses the second patch as training data to train a model for evaluating the three-dimensional shape. The second patch generation unit 120 and the training unit 130 then repeat the process until a criterion is met.

[0017] In the model training device 10, the second patch is used for training. The second patch is a subset of the first patch. That is, in each epoch, the data read from the first storage unit 20 is not the three-dimensional data, but the first patch, which is a subset of the three-dimensional data. Therefore, the load when training the model is reduced.

[0018] A detailed example of the model training device 10 will be described below.

[0019] Fig. 2 is a diagram showing the configuration of model training device 10 together with the environment in which model training device 10 is used. Fig. 3 is a diagram for explaining the processing performed by first patch generating unit 110 of model training device 10. Fig. 4 is a diagram for explaining the processing performed by second patch generating unit 120 of model training device 10.

[0020] In the example shown in FIG. 2, a model training device 10 is used together with a first storage unit 20, a second storage unit 30, and an evaluation device 40.

[0021] The first storage unit 20 stores the first patches. When the model training device 10 generates multiple first patches, the first storage unit 20 stores these multiple first patches. The first storage unit 20 also stores the models trained by the model training device 10.

[0022] The second storage unit 30 stores at least one piece of 3D data used by the model training device 10. As described above, the 3D data represents the shape of an object. The object may be, for example, but is not limited to, a structure such as a bridge or a building, or a part of a structure such as a bridge pier or a bridge girder. The second storage unit 30 may store 3D data for each of multiple objects.

[0023] The three-dimensional data is, for example, point cloud data generated using LiDAR. In this case, the three-dimensional data is a set of points with coordinate data that can be converted into XYZ coordinates. The coordinate data used here may directly indicate XYZ coordinates, or may be polar coordinate data (a set of elevation angle, horizontal angle, and distance) with the sensor position as the origin. If the target object is a large structure such as a bridge or building, the number of points contained in the point cloud data may exceed 10 million.

[0024] The three-dimensional data may be data other than point cloud data, for example, mesh data that represents three-dimensional information as a set of vertices, edges, and faces.

[0025] The three-dimensional data may also have labels set for each of multiple parts within the object. For example, if the three-dimensional data is point cloud data, a label is set for each of the multiple points that make up the point cloud data. An example of the label used here is whether or not the part is abnormal. However, the unit of data to which the label is assigned and the content of the label are not limited to the above example.

[0026] The evaluation device 40 uses the model trained by the model training device 10 to process three-dimensional data of the object to be evaluated.

[0027] As described above, the model training device 10 includes the first patch generation unit 110, the second patch generation unit 120, and the training unit 130. Of these, the first patch generation unit 110 performs processing before training the model, while the second patch generation unit 120 and the training unit 130 perform processing for each epoch.

[0028] As shown in FIG. 3, the first patch generation unit 110 reads three-dimensional data from the second storage unit 30 and generates at least one first patch using the read three-dimensional data. The first patch is a subset of the three-dimensional data and represents the shape of a portion of the object. The first patch generation unit 110 preferably generates multiple first patches from one piece of three-dimensional data. Furthermore, if the second storage unit 30 stores multiple pieces of three-dimensional data, the first patch generation unit 110 generates at least one first patch, preferably multiple first patches, from each of the multiple pieces of three-dimensional data.

[0029] The amount of data in the first patch is, for example, but not limited to, 1 / 100 or less of the three-dimensional data used to generate the first patch. If the three-dimensional data is point cloud data, the number of points included in the first patch is, for example, but not limited to, 100,000 or less.

[0030] The first patch generating unit 110 preferably ensures that all parts of the three-dimensional data are included in at least one first patch, although at least some of the three-dimensional data may be included in multiple first patches.

[0031] Then, the first patch generating unit 110 stores the generated first patch in the first storage unit 20. Here, when the first patch generating unit 110 generates a first patch from each of a plurality of three-dimensional data, it is preferable that the first patch generating unit 110 stores the first patch in the first storage unit 20 in association with the three-dimensional data used when generating the first patch.

[0032] The first patch generation unit 110 may generate the first patch using labels included in the three-dimensional data. For example, there may be a large difference in the number of data items having the label between labels. In this case, it is preferable that the first patch generation unit 110 generates the first patch so that the content of the training data is not biased due to this difference.

[0033] For example, consider a case where 3D data includes multiple parts, each of which is labeled. A first number, which is the number of parts with a first label, is assumed to be smaller than a second number, which is the number of parts with a second label. If the number of first patches containing the first label is the first patch number and the number of first patches containing the second label is the second patch number, the first patch generation unit 110 sets the ratio of the first number of patches to the second number of patches higher than the ratio of the first number to the second number. This increases the amount of training data with the first label compared to when nothing is done.

[0034] As an example, consider a case where a label indicates whether or not the portion contains an anomaly. In this case, there is often less anomalous data than non-anomalous data. In this case, a label indicating an anomaly corresponds to the first label described above, and a label indicating a non-anomaly corresponds to the second label described above. The first patch generation unit 110 preferentially generates first patches including data labeled as an anomaly. In other words, the first patch generation unit 110 leaves first patches corresponding to minority labels (e.g., the first label described above) with a higher probability than first patches corresponding to majority labels (e.g., the second label described above).

[0035] As an example, the operation of the first patch generation unit 110 in semantic segmentation where the three-dimensional data is point cloud data and the number of labels is two will be described. It is assumed that label 0 or label 1 is assigned to each point of the three-dimensional data, and label 0 is more prevalent than label 1. In this case, the first patch generation unit 110 operates to retain first patches that include points with label 1 with a higher probability than first patches that do not include label 1. The first patch generation unit 110 may retain only first patches that include label 1.

[0036] Returning to Fig. 2, as shown in Fig. 4, the second patch generation unit 120 reads out the first patches from the first storage unit 20 and generates at least one second patch using the read out first patches. As described with reference to Fig. 1, the second patch is a subset of the first patch and indicates a further part of the shape of the part of the object indicated by the first patch.

[0037] The second patch generating unit 120 may generate the second patch by any method as long as it can generate patches as training data. Examples of the method that can be used here include the following (1) to (3).

[0038] (1) The second patch generating unit 120 overlays a predetermined shape on the first patch and generates a second patch using the result of the overlay. The predetermined shape is, for example, a convex shape such as a sphere, a rectangular parallelepiped, a cylinder, or an ellipsoid, or a union or difference set of multiple convex shapes. The second patch generation unit 120 then determines, for example, the intersection between the first patch and this predetermined shape as the second patch. Here, the portion of the first patch that overlaps with the predetermined shape and the orientation of the predetermined shape may be selected randomly.

[0039] (2) The second patch generating unit 120 generates the second patch by selecting a reference point in the first patch and selecting other portions from the reference point according to a predetermined rule. For example, if the three-dimensional data is point cloud data, the first patch and the second patch are also point cloud data. In this case, the second patch generation unit 120 first selects a point to be a reference point from the multiple points included in the first patch. This selection may be performed randomly. Then, the second patch generation unit 120 selects a predetermined number of points from this reference point according to a predetermined rule. Then, the second patch generation unit 120 generates the second patch from these points and the reference point.

[0040] Here, when selecting points other than the reference points, the second patch generating unit 120 can use, for example, the following two methods (2-1) and (2-2).

[0041] (2-1) The second patch generation unit 120 selects points that are closest in distance to the reference point. Here, the second patch generation unit 120 may select a predetermined number of points in order of proximity to the reference point. This technique is, for example, K-nearest neighbor search.

[0042] (2-2) The second patch generation unit 120 selects points that are close in distance and characteristics to the reference point. Here, the second patch generation unit 120 may randomly select the reference point. Then, the second patch generation unit 120 may select points whose similarity to the reference point in shape information such as normals and PCA features satisfies a criterion, and select a predetermined number of points from these points in order of proximity to the reference point. This technique is, for example, a region growing method.

[0043] (3) The second patch generating unit 120 generates second patches by dividing the first patch into a plurality of subsets that are similar to each other in at least one of distance and shape. The second patch generation unit 120 generates the second patch from the first patch using a segmentation technique, for example, the RANSAC method or the Cut pursuit method. The RANSAC method is a technique for extracting planar parts from three-dimensional data. The Cut pursuit method is a technique for dividing three-dimensional data into multiple subsets that are similar in distance and shape. For example,<https: / / github.com / loicland / cut-pursuit> A detailed example is disclosed in. Then, the subset that will become the second patch is selected randomly.

[0044] As shown in FIG. 4, the training unit 130 trains a model using the second patches generated by the second patch generating unit 120. The model used here is a model for recognizing the state of a three-dimensional shape. The recognition task used in this model is, for example, at least one of class classification, semantic segmentation, and autoencoder, but is not limited to these. The recognition model is, for example, PointNet++, but is not limited to this. A detailed example of PointNet++ can be found at the following URL,<https: / / arxiv.org / abs / 1706.02413> is disclosed in.

[0045] In the training data used by the training unit 130, the explanatory variables are the second patches described above, and the objective variables are set using, for example, labels included in the second patches that serve as explanatory variables. For example, if a label indicates whether or not an item is abnormal, the label becomes the explanatory variable. If the three-dimensional data, the first patch, and the second patch are point cloud data, an example of the explanatory variable is whether or not the second patch includes a point labeled as abnormal.

[0046] The second patch generating unit 120 and the training unit 130 repeat the process until a criterion is met. The criterion used here is, for example, that the number of repetitions (number of epochs) reaches a predetermined value, that the value of the loss function of the model after training satisfies a predetermined condition, or that the accuracy of the model after training satisfies a predetermined condition. However, other criteria may also be used.

[0047] In addition, when the objective variable of the training data is whether or not the second patch, which is the explanatory variable, contains an abnormality, the model trained by the training unit 130 becomes a model for detecting abnormalities in an object, for example, a structure.

[0048] The training unit 130 then stores the trained model in the first storage unit 20. The model training device 10 transmits the model stored in the first storage unit 20 to the evaluation device 40 as needed. The evaluation device 40 uses this model to evaluate the object.

[0049] Next, a detailed example of the operation of the first patch generating section 110 and the second patch generating section 120 will be described with reference to FIGS.

[0050] 5 is a diagram illustrating an example of a problem that may arise from using the first patch generating unit 110. If the conditions used when generating the first patches are not set appropriately, as shown in FIG. 5, there is a possibility that a part of the three-dimensional data will become an area that is not included in any of the first patches.

[0051] FIG. 6 is a diagram illustrating an example of a problem that may arise when using the second patch generation unit 120. If the conditions used to generate the second patch are not set appropriately, the second patch may have a data structure that would not occur if the patch were generated directly from 3D data, as shown in FIG. 6. In the example shown in FIG. 6, the area on the lower right side of the second patch should contain data because it contains part of the object. However, in the example shown in this figure, this area is outside the first patch, and therefore does not contain any data.

[0052] Figures 7(A) and 7(B) are diagrams for explaining the conditions for solving the problem shown in Figure 6. To solve the problem shown in Figure 6, all parts of the second patch must be included in the first patch. The conditions necessary for this are explained below.

[0053] The first patch generation unit 110 randomly generates a first patch from the three-dimensional data, and the second patch generation unit 120 randomly generates a second patch from the first patch. At this time, the first patch generation unit 110 and the second patch generation unit 120 use a parameter that controls randomness. For example, if the first patch and the second patch are both spheres, this parameter is the center position of the sphere. Then, as shown in FIG. 7(A), if the radius of the second patch is r and the radius of the first patch is r + Δr, when the center of the second patch is located within the sphere of radius Δr from the center of the first patch, all of the second patch will be included in the first patch.

[0054] To generalize this, the second patch generation unit 120 must "select a parameter from the range Param(Pm) of parameters that the second patch can take, under the condition that the second patch is contained in the first patch, when selecting the second patch from the mth first patch." Here, an example of Param(Pm) is "a sphere with a radius Δr from the center of the first patch." Note that Param(Pm) may indicate the range that the parameter can take, under the condition that the second patch generated from the mth first patch can also be directly generated from three-dimensional data, as will be described with reference to FIG. 7(C).

[0055] FIG. 7(C) is a diagram illustrating conditions for solving the problem shown in FIG. 5. As shown in this figure, the first patch generation unit 110 needs to ensure that every part of the three-dimensional data is included in at least one first patch. To achieve this, the first patch generation unit 110 needs to generate multiple first patches at intervals d so as to cover the entire three-dimensional data. In this case, d is equal to or smaller than (2 / √3)Δr. Furthermore, as shown in FIG. 7(B), if the center of the second patch is positioned within a sphere of radius Δr from the center of the first patch, the second patch that can be generated will be comparable to a patch generated directly from the three-dimensional data in terms of randomness and coverage.

[0056] This can be generalized as follows: ·Let X' be the 3D data or a subset of the 3D data. The set of parameters that satisfy the condition that the second patch can be used as training data and contains at least a part of X' is defined as PX'. This PX' is a set of parameters such that "when generating a patch from X, the generated patch can be used as training data and contains at least a part of X'." For the mth first patch, the range of parameters that the second patch can take is the above-mentioned Param(Pm). Param(Pm) indicates the range of parameters that the second patch generated from the mth first patch can take when the condition that it can be generated directly from 3D data is satisfied. In this case, the first patch generating unit 110 Param(P1)∪Param(P2)∪...∪Param(PN)⊇PX´···(1) In other words, when the union of Param(Pm) is calculated between m=1 and N, PX' is included in the union. The above formula (1) indicates that "all patches generated from X based on PX' are covered by the union of all patches generated from X based on each Param."

[0057] In the example shown in Figure 7(B), when selecting a parameter from Param(Pm), i.e., the center of the sphere, it is preferable to select the parameter from a cube inscribed in the sphere that is Param(Pm). In this way, there is no overlap between adjacent Param(Pm), and the ease of selecting a parameter can be made uniform across the entire 3D data.

[0058] This can be generalized as follows: When the second patch generating unit 120 generates a second patch from the m-th first patch, if the parameters of the second patch are selected from Param(Pm)', which is a subset of Param(Pm), then: Param(P1)´∩Param(P2)´∩...Param(PN)´=empty set It is preferable to satisfy the following. In other words, when the intersection of Param(Pm)' is calculated between m = 1 and N, this intersection is an empty set. In this way, the second patch generating unit 120 can uniformly select parameters for the second patch from PX'.

[0059] The contents explained using the diagrams in FIG. 7 can be logically explained as follows.

[0060] The second patch generated by the second patch generating unit 120 based on an arbitrary first patch P and an arbitrary parameter p is defined as Q(p, P).

[0061] For any first patch P, a set of parameters p for which Q(p,P) and Q(p,X) are the same is defined as Param(P). In other words, as long as the second patch generation unit 120 generates a second patch based on parameters p included in Param(P), a unique second patch such as that shown in FIG. 6 will not be generated.

[0062] The set of parameters p that satisfy the conditions that Q(p,X) can be used as training data and contains at least a part of X' is denoted as PX'. In other words, the parameter p being included in PX' is equivalent to Q(p,X) being "a second patch that contains at least a part of X'."

[0063] The first patch generation unit 110 generates at least N first patches P1, P2, ..., PN. At this time, the first patch generation unit 110 generates these N first patches by using the above formula (1), i.e., Param(P1)∪Param(P2)∪...∪Param(PN)⊇PX´ is generated so that

[0064] Furthermore, the second patch generating unit 120 selects at least one parameter p included in Param(P) based on the first patch P, and generates at least one second patch Q(p,P).

[0065] In this case, the generated second patch can cover "second patches that include at least a part of X'". This is because, according to the definition of Param(P), the second patch Q(p,P) is always the same as Q(p,X), and furthermore, according to the above formula (1), the selected parameter p can cover PX'.

[0066] 8 is a diagram showing an example of the hardware configuration of the model training device 10. The model training device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0067] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0068] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0069] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0070] The storage device 1040 is an auxiliary storage device realized by removable media such as a hard disk drive (HDD), a solid state drive (SSD), or a memory card, or a read-only memory (ROM), and has a recording medium. The recording medium of the storage device 1040 stores program modules that realize each function of the model training apparatus 10 (e.g., the first patch generation unit 110, the second patch generation unit 120, and the training unit 130). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also function as at least one of the first memory unit 20 and the second memory unit 30.

[0071] The input / output interface 1050 is an interface for connecting the model training device 10 to various input / output devices. For example, the model training device 10 communicates with the first storage unit 20 and the second storage unit 30 via the input / output interface 1050.

[0072] Network interface 1060 is an interface for connecting model training device 10 to a network. This network may be, for example, a local area network (LAN) or a wide area network (WAN). Network interface 1060 may connect to the network wirelessly or via a wired connection. Model training device 10 may communicate with evaluation device 40 via network interface 1060.

[0073] 9 is a diagram showing a first example of processing performed by the first patch generating unit 110 of the model training device 10. The first patch generating unit 110 generates a first patch in advance before the second patch generating unit 120 and the training unit 130 perform processing.

[0074] Specifically, the first patch generation unit 110 reads out the three-dimensional data to be processed from the second storage unit 30 (step S10). Next, the first patch generation unit 110 generates a first patch from the read out three-dimensional data (step S20) and stores the generated first patch in the first storage unit 20 (step S30). The first patch generation unit 110 repeats steps S20 and S30 until the number of first patches stored in the first storage unit 20 satisfies a criterion (step S40).

[0075] When the second storage unit 30 stores a plurality of three-dimensional data, the first patch generating unit 110 performs the process shown in FIG. 9 on each of the plurality of three-dimensional data.

[0076] 10 is a diagram showing a second example of processing performed by the first patch generation unit 110 of the model training device 10. In the example shown in this figure, the first patch generation unit 110 is similar to the example shown in FIG. 9 except that it generates a first patch using labels included in the three-dimensional data (step S22). A specific example of how to use the labels is as explained using FIG. 2.

[0077] 11 is a diagram showing an example of processing performed by the second patch generation unit 120 and the training unit 130 of the model training device 10. First, the second patch generation unit 120 reads a first patch from the first storage unit 20. Here, if the first storage unit 20 stores multiple first patches, the second patch generation unit 120 selects one first patch from these multiple first patches and reads the selected first patch. At this time, the second patch generation unit 120 randomly selects a first patch (step S110).

[0078] Next, the second patch generating unit 120 generates at least one second patch using the read first patch (step S120). Next, the training unit 130 trains a model using the second patch generated by the second patch generating unit 120, and stores the trained model in the first storage unit 20 (step S130).

[0079] The second patch generating unit 120 and the training unit 130 repeat the processes shown in steps S110 to S130 until the criterion is met (step S140).

[0080] As described above, according to this embodiment, the model training device 10 includes a first patch generation unit 110. The first patch generation unit 110 generates first patches, which are subsets of the three-dimensional data, and stores them in the first storage unit 20. The second patch generation unit 120 then uses the first patches stored in the first storage unit 20 to generate second patches, which serve as training data. The first patches are 3 Since the training data is a subset of three-dimensional data, the load on the model training device 10 is smaller than when training patches directly from three-dimensional data. Therefore, the load on the model training device 10 when training a model to evaluate three-dimensional data that indicates the shape of an object is smaller.

[0081] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0082] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0083] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. A first patch generating means for generating at least one first patch, which is a subset of three-dimensional data representing the shape of an object, using the three-dimensional data, and storing the first patch in a storage means; second patch generating means for reading any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training means for training a model for evaluating a three-dimensional shape using the second patch as training data; Equipped with The second patch generating means and the training means repeat the process until a criterion is met. 2. In the model training device described in 1 above, the first patch generation means generates the N first patches using a parameter for controlling randomness of the patches; the second patch generating means generates the second patch using the parameters; The three-dimensional data or a subset of the three-dimensional data is defined as X', The set of parameters that satisfies the condition that the second patch is available as training data and includes at least a part of X' is defined as PX', For the m-th first patch, when the condition that the second patch generated from the first patch can be generated directly from the three-dimensional data is satisfied, and the range that the parameter can take is Param(Pm), The first patch generation means calculates a union of the Param(Pm) between m=1 and N, and the first patch generation means causes the PX′ to be included in the union. 3. In the model training device described in 2 above, The second patch generation means selects the parameters from the Param (Pm). 4. In the model training device according to 2 or 3 above, the second patch generating means, when generating the second patch from the m-th first patch, selects parameters of the second patch from Param(Pm)', which is a subset of Param(Pm); A model training device, wherein when the intersection set of Param(Pm)′ is calculated between m=1 and N, the intersection set becomes an empty set. 5. In the model training device according to any one of 1 to 4 above, the three-dimensional data has labels set for each of a plurality of parts within the object; The first patch generation means generates a plurality of the first patches using the labels. 6. In the model training device described in 5 above, a first number, which is the number of the portions having the first label, being less than a second number, which is the number of the portions having the second label; When the number of the first patches including the first label is defined as a first patch number, and the number of the first patches including the second label is defined as a second patch number, The first patch generation means sets a ratio of the first number of patches to the second number of patches higher than a ratio of the first number to the second number. 7. In the model training device according to any one of 1 to 6 above, The second patch generation means overlays a predetermined shape on the first patch and generates the second patch using the overlay result. 8. In the model training device according to any one of 1 to 6 above, The second patch generation means generates the second patch by selecting a reference point in the first patch and selecting another portion from the first patch from the reference point according to a predetermined rule. 9. In the model training device according to any one of 1 to 6 above, The second patch generation means generates the second patches by dividing the first patches into a plurality of subsets that are similar to each other in at least one of distance and shape. 10. In the model training device according to any one of 1 to 9 above, the training data includes information indicating whether the second patch includes an abnormality that has occurred in the object; and The model training device, wherein the model is a model for detecting an abnormal portion of an object. 11. The computer generating at least one first patch that is a subset of the three-dimensional data representing the shape of the object, and storing the first patch in a storage means; reading out any of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training a model for evaluating a three-dimensional shape using the second patch as training data; A model training method that repeats the generation of the second patch and the training of the model until a criterion is met. 12. The model training method according to claim 11, the computer generates the N first patches using a parameter for controlling the randomness of the patches; generating the second patch using the parameters; The three-dimensional data or a subset of the three-dimensional data is defined as X', The set of parameters that satisfies the condition that the second patch is available as training data and includes at least a part of X' is defined as PX', For the m-th first patch, when the condition that the second patch generated from the first patch can be generated directly from the three-dimensional data is satisfied, and the range that the parameter can take is Param(Pm), The model training method, wherein when the computer calculates a union of the Param(Pm) between m=1 and N, the computer ensures that the PX′ is included in the union. 13. The model training method according to claim 12, A model training method in which the computer selects the parameters from the Param (Pm). 14. The model training method according to 12 or 13 above, When generating the second patch from the m-th first patch, the computer selects parameters for the second patch from Param(Pm)', which is a subset of Param(Pm); A model training method, wherein when the intersection set of Param(Pm)' is calculated between m=1 and N, the intersection set becomes an empty set. 15. In the model training method according to any one of 11 to 14 above, the three-dimensional data has labels set for each of a plurality of parts within the object; The computer generates a plurality of the first patches using the labels. 16. The model training method according to claim 15, a first number, which is the number of the portions having the first label, being less than a second number, which is the number of the portions having the second label; When the number of the first patches including the first label is defined as a first patch number, and the number of the first patches including the second label is defined as a second patch number, The computer sets a ratio of the first number of patches to the second number of patches higher than a ratio of the first number to the second number. 17. In the model training method according to any one of claims 11 to 16, The computer overlays a predetermined shape onto the first patch and generates the second patch using the overlay result. 18. In the model training method according to any one of the above items 11 to 16, The computer generates the second patch by selecting a reference point within the first patch and selecting another portion from the first patch from the reference point according to a predetermined rule. 19. In the model training method according to any one of claims 11 to 16, The computer generates the second patches by dividing the first patches into a plurality of subsets that are similar to each other in at least one of distance and shape. 20. In the model training method according to any one of the above items 11 to 19, the training data includes information indicating whether the second patch includes an abnormality that has occurred in the object; and A model training method, wherein the model is a model for detecting an abnormal portion in an object. 31. To the computer, a first patch generation function that uses three-dimensional data indicating the shape of an object to generate at least one first patch that is a subset of the three-dimensional data and stores the first patch in a storage means; a second patch generation function that reads out any one of the first patches from the storage means and generates at least one second patch that is a subset of the first patch; a training function for training a model for evaluating a three-dimensional shape using the second patch as training data; It records the program that allows you to The second patch generation function and the training function repeat the process until a criterion is met. 32. The recording medium according to claim 31, the first patch generation function generates N first patches using a parameter for controlling the randomness of the patches; the second patch generation function generates the second patch using the parameters; The three-dimensional data or a subset of the three-dimensional data is defined as X', The set of parameters that satisfies the condition that the second patch is available as training data and includes at least a part of X' is defined as PX', For the m-th first patch, when the condition that the second patch generated from the first patch can be generated directly from the three-dimensional data is satisfied, and the range that the parameter can take is Param(Pm), The first patch generating function calculates a union of the Param (Pm) between m=1 and N, and causes the PX′ to be included in the union. 33. The recording medium according to claim 32, A recording medium, wherein the second patch generation function selects the parameters from the Param (Pm). 34. The recording medium according to 32 or 33 above, the second patch generation function, when generating the second patch from the m-th first patch, selects parameters of the second patch from Param(Pm)', which is a subset of Param(Pm); A recording medium, wherein when the intersection set of Param(Pm)' is calculated between m=1 and N, the intersection set becomes an empty set. 35. The recording medium according to any one of the above items 31 to 34, the three-dimensional data has labels set for each of a plurality of parts within the object; a recording medium, wherein the first patch generation function generates a plurality of the first patches using the labels; 36. The recording medium according to claim 35, a first number, which is the number of the portions having the first label, being less than a second number, which is the number of the portions having the second label; When the number of the first patches including the first label is defined as a first patch number, and the number of the first patches including the second label is defined as a second patch number, The first patch generation function sets a ratio of the first number of patches to the second number of patches higher than a ratio of the first number to the second number. 37. The recording medium according to any one of the above items 31 to 36, The second patch generation function overlays a predetermined shape on the first patch and generates the second patch using the result of the overlay. 38. The recording medium according to any one of the above items 31 to 36, A recording medium in which the second patch generation function generates the second patch by selecting a reference point within the first patch and selecting other parts from the first patch from the reference point according to a predetermined rule. 39. The recording medium according to any one of the above items 31 to 36, The second patch generation function generates the second patch by dividing the first patch into a plurality of subsets that are similar to each other in at least one of distance and shape. 40. The recording medium according to any one of the above items 31 to 39, the training data includes information indicating whether the second patch includes an abnormality that has occurred in the object; and The recording medium, wherein the model is a model for detecting an abnormal portion of an object. 41. A program according to any one of items 31 to 40 above. [Explanation of symbols]

[0084] 10 Model training device 110 First patch generation unit 120 Second patch generation unit 130 Training Department 20 1st memory section 30 2nd memory section 40 Evaluation equipment

Claims

1. a first patch generating means for generating at least one first patch, which is a subset of three-dimensional data representing the shape of an object, using the three-dimensional data, and storing the first patch in a storage means; second patch generating means for reading any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training means for training a model for evaluating a three-dimensional shape using the second patch as training data; Equipped with the second patch generating means and the training means repeat the process until a criterion is met; the first patch generation means generates N first patches using a parameter for controlling randomness of the patches; the second patch generating means generates the second patch using the parameters; The three-dimensional data or a subset of the three-dimensional data is defined as X', The set of parameters P X′ is defined as the second patch that satisfies the condition that the second patch is available as training data and includes at least a part of X′, For the m-th first patch, when the condition that the second patch generated from the first patch can be generated directly from the three-dimensional data is satisfied, and the range that the parameter can take is Param(Pm), the first patch generation means calculates a union of the Param (Pm) between m=1 and m=N, so that the PX′ is included in the union; the second patch generating means, when generating the second patch from the m-th first patch, selects parameters of the second patch from Param(Pm)', which is a subset of Param(Pm); When the intersection set of Param(Pm)' is calculated between m = 1 and N, the intersection set becomes an empty set.

2. a first patch generating means for generating at least one first patch, which is a subset of three-dimensional data representing the shape of an object, using the three-dimensional data, and storing the first patch in a storage means; second patch generating means for reading any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training means for training a model for evaluating a three-dimensional shape using the second patch as training data; Equipped with the second patch generating means and the training means repeat the process until a criterion is met; the three-dimensional data has labels set for each of a plurality of parts within the object; the first patch generating means generates a plurality of the first patches using the labels; a first number, which is the number of the portions having a first of the labels, is less than a second number, which is the number of the portions having a second of the labels; When the number of the first patches including the first label is defined as a first patch number, and the number of the first patches including the second label is defined as a second patch number, The first patch generation means sets a ratio of the first number of patches to the second number of patches higher than a ratio of the first number to the second number.

3. 2. The model training device according to claim 1, The second patch generation means selects the parameters from the Param (Pm).

4. The model training device according to any one of claims 1 to 3, The second patch generation means generates the second patches by dividing the first patches into a plurality of subsets that are similar to each other in at least one of distance and shape.

5. The model training device according to any one of claims 1 to 4, the training data includes information indicating whether the second patch includes an abnormality that has occurred in the object; The model training device, wherein the model is a model for detecting an abnormal portion of an object.

6. The computer generating at least one first patch that is a subset of the three-dimensional data representing the shape of the object, and storing the first patch in a storage means; reading out any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training a model for evaluating a three-dimensional shape using the second patch as training data; repeating the generation of the second patch and training of the model until a criterion is met; When generating the first patches, N first patches are generated using a parameter for controlling the randomness of the patches; When generating the second patch, generate the second patch using the parameters; The three-dimensional data or a subset of the three-dimensional data is defined as X', The set of parameters P X′ is defined as the second patch that satisfies the condition that the second patch is available as training data and includes at least a part of X′, For the m-th first patch, when the condition that the second patch generated from the first patch can be generated directly from the three-dimensional data is satisfied, and the range that the parameter can take is Param(Pm), When generating the first patch, when a union of the Param (Pm) is calculated between m = 1 and N, the PX' is included in the union; When generating the second patch from the m-th first patch, select parameters of the second patch from Param(Pm)', which is a subset of Param(Pm); A model training method in which, when the intersection set of Param(Pm)' is calculated between m = 1 and N, the intersection set becomes an empty set.

7. The computer generating at least one first patch that is a subset of the three-dimensional data representing the shape of the object, and storing the first patch in a storage means; reading out any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; training a model for evaluating a three-dimensional shape using the second patch as training data; repeating the generation of the second patch and training of the model until a criterion is met; the three-dimensional data has labels set for each of a plurality of parts within the object; When generating the first patch, generate a plurality of the first patches using the label; a first number, which is the number of the portions having a first of the labels, is less than a second number, which is the number of the portions having a second of the labels; When generating the first patch, When the number of the first patches including the first label is defined as a first patch number, and the number of the first patches including the second label is defined as a second patch number, a ratio of the first number of patches to the second number of patches is higher than a ratio of the first number to the second number.

8. On the computer, a first patch generation function that uses three-dimensional data indicating the shape of an object to generate at least one first patch that is a subset of the three-dimensional data and stores the first patch in a storage means; a second patch generating function for reading any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; a training function for training a model for evaluating a three-dimensional shape using the second patch as training data; Let them have the second patch generation function and the training function repeat the process until a criterion is met; the first patch generation function generates N first patches using a parameter for controlling randomness of the patches; the second patch generation function generates the second patch using the parameters; The three-dimensional data or a subset of the three-dimensional data is defined as X', The set of parameters P X′ is defined as the second patch that satisfies the condition that the second patch is available as training data and includes at least a part of X′, For the m-th first patch, when the condition that the second patch generated from the first patch can be generated directly from the three-dimensional data is satisfied, and the range that the parameter can take is Param(Pm), The first patch generation function calculates a union of the Param (Pm) between m = 1 to N so that the PX' is included in the union, the second patch generation function, when generating the second patch from the m-th first patch, selects parameters of the second patch from Param(Pm)′, which is a subset of Param(Pm); A program in which, when the intersection set of Param(Pm)' is calculated between m = 1 and N, the intersection set becomes an empty set.

9. On the computer, a first patch generation function that uses three-dimensional data indicating the shape of an object to generate at least one first patch that is a subset of the three-dimensional data and stores the first patch in a storage means; a second patch generating function for reading any one of the first patches from the storage means and generating at least one second patch that is a subset of the first patch; a training function for training a model for evaluating a three-dimensional shape using the second patch as training data; Let them have the second patch generation function and the training function repeat the process until a criterion is met; the three-dimensional data has labels set for each of a plurality of parts within the object; the first patch generation function generates a plurality of the first patches using the labels; a first number, which is the number of the portions having a first of the labels, is less than a second number, which is the number of the portions having a second of the labels; When the number of the first patches including the first label is defined as a first patch number, and the number of the first patches including the second label is defined as a second patch number, The program, wherein the first patch generation function sets a ratio of the first number of patches to the second number of patches higher than a ratio of the first number to the second number.

Images