Deformation composite data generation device and deformation composite data generation method

Abstract

A deformation composition data generation apparatus 20 comprises: an acquisition unit 21 that acquires the distribution of the amounts of displacement with respect to points in point cloud data; and a composition unit 22 that composes the amounts of displacement in accordance with the distribution of the points in the point cloud data.

Description

[Technical Field]

[0001] The present invention relates to a deformation synthetic data generation device, a deformation synthetic data generation method, and a deformation synthetic data generation program, and in particular to a deformation synthetic data generation device, a deformation synthetic data generation method, and a deformation synthetic data generation program that generate point cloud data having deformations such as damage. [Background technology]

[0002] Improving the efficiency of inspection work for infrastructure facilities such as bridges, tunnels, and concrete structures is a social challenge. Attempts are being made to apply machine learning or deep learning to detect damage and anomalies such as cracks, peeling, and exposed rebar.

[0003] Generally, in order to apply machine learning or deep learning, it is necessary to prepare a large amount of training data. Because it is costly to prepare a large amount of training data from a state where there is no data to generate it from, technologies have been devised to artificially generate new training data from existing data.

[0004] Images are often used as measurement information for analyzing the above-mentioned deformations. Point cloud data obtained by LiDAR (Light Detection and Ranging) and other measurement information is also attracting attention. Point cloud data retains the coordinates of three-dimensional space as they are. Therefore, point cloud data is thought to be more effective information for analyzing deformations including three-dimensional deformations.

[0005] Furthermore, Non-Patent Document 1 describes a technology that uses a generative adversarial network (GAN), a type of deep learning model, to generate a variety of pseudo-crack images similar to real data from annotation images of cracks.

[0006] Furthermore, Non-Patent Document 2 describes a technique for extracting a detection target object from an image and pasting the extracted object onto another image to generate a variety of learning data.

[0007] Furthermore, Non-Patent Document 3 describes a technique for extracting geometric structures such as planes and cylinders from point cloud data using random sample consensus (RANSAC).

[0008] Furthermore, Non-Patent Document 4 describes a technology related to PointNet++, which is a type of deep learning network that uses point cloud data as input. [Prior art documents] [Non-patent literature]

[0009] [Non-Patent Document 1] Tomotaka Fukuoka, Takahiro Minami, Wataru Urata, Makoto Fujio, and Junichi Takayama, "Generating Pseudo Training Data Using Pix2Pix for Bridge Inspection Using Deep Learning," Transactions of the Japan Society of Civil Engineers, F4 (Construction Management), Vol. 75, No. 2, I_27-I_35, 2019. [Non-patent document 2] Debidatta Dwibedi, Ishan Misra, and Martial Hebert, "Cut, paste and learn: Surprisingly easy synthesis for instance detection," In Proceedings of the IEEE International Conference on Computer Vision, pages 1301-1310, 2017. [Non-patent document 3] R. Schnabel, R. Wahl, and R. Klein, "Efficient RANSAC for point-cloud shape detection," In Computer Graphics Forum, Citeseer, vol. 26, pages 214-226, 2007. [Non-patent document 4] CR Qi, L. Yi, H. Su, and LJ Guibas, "PointNet++: Deep hierarchical feature learning on point sets in a metric space," In Advances in Neural Information Processing Systems (NIPS), 2017. Summary of the Invention [Problem to be solved by the invention]

[0010] The application range of the training data generation technology using a generative adversarial network (GAN), which is a type of deep learning model, described in Non-Patent Document 1 is limited to images. In other words, it is difficult to apply the training data generation technology described in Non-Patent Document 1 to point cloud data.

[0011] In general, technological development in machine learning or deep learning for point cloud data is not as advanced as that for images, and therefore it is thought to be difficult to extend the training data generation technology using GAN, a type of deep learning model, so that it can also be applied to point cloud data.

[0012] Furthermore, the training data generation technology described in Non-Patent Document 2 is a technology that pastes extracted target objects such as food items onto other images, and does not use machine learning or deep learning.

[0013] However, it is difficult to apply the learning data generation technology described in Non-Patent Document 2 to generating anomalies. The reason is that an anomaly is not an object itself but a deformation of a part of an object, and it is difficult to represent an anomaly simply by arranging the object.

[0014] 18 is an explanatory diagram showing examples of point cloud data with and without deformation. The point cloud data shown in FIG. 18 represents bridge girders, building ceilings, etc.

[0015] The point cloud data shown in the upper part of Fig. 18 is point cloud data without deformation. The point cloud data shown in the lower part of Fig. 18 is point cloud data with deformation. The difference between the two point cloud data shown in Fig. 18 also includes deformation, i.e., movement of points. It is difficult to represent the movement of points by the arrangement of objects.

[0016] Furthermore, Non-Patent Documents 3 and 4 do not describe generating training data from point cloud data.

[0017] Therefore, one of the objects of the present invention is to provide a deformation composite data generation device, a deformation composite data generation method, and a deformation composite data generation program that can generate new data containing deformations even if the data format is a point cloud. [Means for solving the problem]

[0018] The deformation composite data generation device according to the present invention includes an acquisition unit that acquires a distribution of displacement amounts for points of point cloud data, and a synthesis unit that synthesizes displacement amounts according to the distribution of points in the point cloud data. The distribution is a distribution of displacement of each point in the point cloud data from a model representing a shape formed by the point cloud data in an arbitrary range of the point cloud data, and the apparatus further includes a selection unit that selects an arbitrary range of the point cloud data to be synthesized, and a generation unit that generates a model for the arbitrary range selected in the point cloud data, the model representing a spherical or cylindrical geometric structure, and the generation unit generates the model by calculating an equation of the sphere or cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure. It is characterized by:

[0019] The deformation composite data generation method according to the present invention acquires the distribution of displacement amounts for points in point cloud data, and synthesizes the displacement amounts according to the distribution of points in the point cloud data. The distribution is a distribution of displacement of each point in the point cloud data from a model that represents the shape formed by the point cloud data in an arbitrary range of the point cloud data, and an arbitrary range to be synthesized is selected for the point cloud data, and a model for the arbitrary range selected in the point cloud data is generated, the model representing a spherical or cylindrical geometric structure, and when generating the model, the model is generated by calculating an equation of the sphere or cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure. It is characterized by:

[0020] The deformation composite data generation program according to the present invention causes a computer to execute an acquisition process for acquiring a distribution of displacement amounts for points of point cloud data, and a synthesis process for synthesizing displacement amounts according to the distribution of points in the point cloud data. The distribution is a distribution of displacement of each point in the point cloud data from a model representing the shape formed by the point cloud data in an arbitrary range of the point cloud data, and the computer executes a process of selecting an arbitrary range in which synthesis is performed for the point cloud data and a process of generating a model for the arbitrary range selected in the point cloud data, the model representing a spherical or cylindrical geometric structure, and when generating the model, the computer generates the model by calculating an equation of the sphere or cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters characterizing the geometric structure. It is characterized by: [Effects of the Invention]

[0021] According to the present invention, even if the data format is a point cloud, data having deformations can be newly generated. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a block diagram showing an example of the configuration of a deformation composite data generation device according to a first embodiment of the present invention. [Figure 2] 10 is an explanatory diagram showing an example of a plane represented by a plane equation calculated by the structural modeling unit 120. FIG. [Figure 3] 1 is a block diagram showing an example of the configuration of a deformation characteristic data generation device according to a first embodiment of the present invention. [Figure 4] 1 is an explanatory diagram showing an example of a deformation characteristic generated by the deformation characteristic data generation device 200. FIG. [Figure 5] 10 is an explanatory diagram showing another example of a deformation characteristic generated by the deformation characteristic data generation device 200. FIG. [Figure 6] 10 is an explanatory diagram showing an example of the synthesis of deformation characteristics by the deformation characteristic synthesis unit 140. FIG. [Figure 7] 10 is a flowchart showing the operation of a deformation feature synthesis process performed by the deformation composite data generation device 100 of the first embodiment. [Figure 8] FIG. 10 is a block diagram showing an example of the configuration of a deformation composite data generation device according to a second embodiment of the present invention. [Figure 9] 10 is a flowchart showing the operation of a deformation feature synthesis process by the deformation composite data generation device 101 of the second embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a deformation composite data generation device according to a third embodiment of the present invention. [Figure 11]FIG. 10 is an explanatory diagram showing an example of a cylinder that the deformation composite data generation device 102 treats as a geometric structure. [Figure 12] 10 is an explanatory diagram showing an example of projection of points belonging to a selected range onto a geometric structure by the structural modeling unit 121. FIG. [Figure 13] 10 is a flowchart showing the operation of a deformation feature synthesis process by the deformation composite data generation device 102 of the third embodiment. [Figure 14] FIG. 10 is a block diagram showing an example of the configuration of a deformation composite data generation device according to a fourth embodiment of the present invention. [Figure 15] 10 is a flowchart showing the operation of a deformation feature synthesis process by the deformation composite data generation device 103 of the fourth embodiment. [Figure 16] 1 is an explanatory diagram showing an example of the hardware configuration of a deformation composite data generation device according to the present invention. [Figure 17] 1 is a block diagram showing an overview of a deformation composite data generation device according to the present invention. [Figure 18] FIG. 10 is an explanatory diagram showing examples of point cloud data with and without deformation. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. Note that the deformation in each embodiment represents a change in state, such as damage such as cracks or breakage.

[0024] [First embodiment] [Configuration Description] Fig. 1 is a block diagram showing an example of the configuration of a deformation composite data generation device according to a first embodiment of the present invention. The deformation composite data generation device 100 shown in Fig. 1 includes a deformation composite range selection unit 110, a structural modeling unit 120, a deformation feature database 130, and a deformation feature synthesis unit 140.

[0025] 1, the deformation characteristic database 130 is communicably connected to the deformation characteristic data generation device 200. Note that the deformation characteristic database 130 does not have to be connected to the deformation characteristic data generation device 200.

[0026] A point cloud, which is a data format to be processed by the deformation composite data generation device 100 shown in Fig. 1, is a set of points having coordinates. The following explanation is based on the assumption that the coordinates are three-dimensional Cartesian coordinates (x, y, z). Note that the coordinates of the points may be expressed in a coordinate system other than the Cartesian coordinate system, such as a polar coordinate system.

[0027] The deformation composite data generation device 100 of this embodiment receives point cloud data to be subjected to deformation synthesis as input data, and outputs data in which deformation is synthesized into the input data. Note that the input data does not necessarily have deformation.

[0028] The composite deformation data generation device 100 of this embodiment is characterized by synthesizing deformations for areas with planar structures. Note that the planar structure may be any structure that is determined to be planar as a result of a process for estimating that the structure is planar, as will be described later, and does not necessarily have to be a strict mathematically defined plane.

[0029] The deformation synthesis range selection unit 110 has a function of selecting the range (set of points) for deformation synthesis for the input point cloud data. For example, the deformation synthesis range selection unit 110 selects the range for deformation synthesis for a planar structure that has been manually identified using point cloud processing software.

[0030] Alternatively, the deformation synthesis range selection unit 110 may select a range for deformation synthesis for a planar structure extracted using an algorithm based on RANSAC described in Non-Patent Document 3. The deformation synthesis range selection unit 110 inputs the input original point cloud data and the selected range to the structural modeling unit 120.

[0031] The structural modeling unit 120 has a function of modeling the selected range input from the deformation synthesis range selection unit 110 on a plane. Specifically, it calculates a plane equation through which points belonging to the selected range input from the deformation synthesis range selection unit 110 pass.

[0032] The structural modeling unit 120 can calculate the plane equation by solving an optimization problem using, for example, the least squares method, etc. The structural modeling unit 120 calculates, for example, the following equation (1) as the plane equation.

[0033]

number

[0034] Note that the girders of many bridges and the ceilings of many structures are nearly parallel to the ground. Also, the walls of many structures are nearly perpendicular to the ground. The structural modeling unit 120 may use the above facts to simplify the calculation of the plane equation.

[0035] Furthermore, if the plane equation has already been calculated when the deformation synthesis range selection unit 110 extracts the planar structure, the structural modeling unit 120 may omit the process of calculating the plane equation.

[0036] Fig. 2 is an explanatory diagram showing an example of a plane represented by the plane equation calculated by the structural modeling unit 120. Next, the structural modeling unit 120 determines the x'-axis and y'-axis constituting a Cartesian coordinate system on the plane represented by the calculated plane equation as shown in Fig. 2. The structural modeling unit 120 can determine the x'-axis and y'-axis as axes representing two directions orthogonal to the normal directions (a, b, c).

[0037] When the plane represents a wall of a structure or the like, the structural modeling unit 120 may obtain the plane based on a predetermined rule that the x' axis is parallel to the ground and the y' axis is perpendicular to the ground.

[0038] Furthermore, when the plane represents a bridge girder or the like, the structural modeling unit 120 may obtain the plane based on a predetermined rule that the x' axis is set in the direction of travel of the bridge, and the y' axis is set in a direction perpendicular to the direction of travel.

[0039] After determining the x'-axis and y'-axis, the structural modeling unit 120 calculates the (x', y') coordinates of each point (black circle shown in Fig. 2) belonging to the selected range when the point is projected onto a plane as shown in Fig. 2. The structural modeling unit 120 inputs the input original point cloud data, the calculated plane equation, and the (x', y') coordinates of each point belonging to the selected range to the deformation feature synthesis unit 140.

[0040] The deformation feature database 130 has the function of storing the deformation features to be synthesized. A deformation feature is a feature quantity of a deformation when a specific feature of the deformation is focused on. For example, the function d(x', y') described below is a deformation feature when a displacement is focused on as a specific feature.

[0041] The deformation characteristics are generated, for example, by a deformation characteristic data generation device 200 shown in Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the deformation characteristic data generation device according to the first embodiment of the present invention.

[0042] 3 includes a deformation extraction range selection unit 210, a structural modeling unit 220, and a deformation feature extraction unit 230. The deformation feature data generation device 200 of this embodiment receives point cloud data containing deformations as input, and stores the extracted deformation features in the deformation feature database 130.

[0043] The deformation extraction range selection unit 210 selects a planar structure area including deformations to be used for synthesis from the input point cloud data containing deformations. The planar structure area may be selected by the user using point cloud processing software.

[0044] Furthermore, if label information is attached to the deformed location, the deformation extraction range selection unit 210 may select the area of ​​the planar structure by combining the label information with the extraction of the planar structure using an algorithm based on RANSAC described in Non-Patent Document 3. The deformation extraction range selection unit 210 inputs the input original point cloud data and the selected range to the structural modeling unit 220.

[0045] Similar to the structural modeling unit 120, the structural modeling unit 220 calculates a plane equation through which the points belonging to the selected range input from the deformation extraction range selection unit 210 pass. Next, the structural modeling unit 220 calculates the (x', y') coordinates of the points belonging to the selected range when projected onto the plane represented by the calculated plane equation. Next, the structural modeling unit 220 inputs the calculated plane equation and the (x', y') coordinates of each point belonging to the selected range to the deformation feature extraction unit 230.

[0046] The deformation feature extraction unit 230 constructs a function d(x', y') that represents the relationship between the (x', y') coordinates and the amount of displacement from a plane, based on the input from the structural modeling unit 220. For example, the deformation feature extraction unit 230 defines the amount of displacement as a signed distance from a plane, and then calculates the function d(x', y') using the original coordinates (x, y, z) as shown in the following formula (2).

[0047]

number

[0048] In addition, the deformation feature extraction unit 230 can construct a function d(x', y') defined in a continuous region by interpolating d(x', y') obtained from points belonging to the selected range using nearest neighbor interpolation or the like.

[0049] 4 and 5 are explanatory diagrams showing examples of deformation characteristics generated by the deformation characteristic data generation device 200. The upper parts of FIGS. 4 and 5 show point cloud data having deformations input to the deformation extraction range selection unit 210.

[0050] The lower part of Figs. 4 to 5 shows the function d(x', y') constructed by the deformation feature extraction unit 230. Note that Figs. 4 to 5 only show the x' direction. As shown in Figs. 4 to 5, in this embodiment, information on the distribution of the amount of displacement from a plane parallel to the x' axis shown in Figs. 4 to 5, expressed as a function of (x', y'), is treated as the deformation feature.

[0051] The deformation features shown in Fig. 4 are generated from point cloud data having a deformation of "uplift." The deformation features shown in Fig. 5 are generated from point cloud data having a deformation of "subsidence."

[0052] If each point has label information, such as information on the presence or absence of deformation, the type of deformation, or the scale of deformation, it is preferable to perform synthesis including the label information. If each point has label information, the deformation feature extraction unit 230 also defines a function L(x', y') representing the label information using interpolation or the like, in the same way as the function d(x', y').

[0053] Examples of the type and scale of deformation include the classification of damage such as cracks, peeling, exposed rebar, and fissures, as well as the assessment categories that indicate the degree of damage, as stipulated in the Ministry of Land, Infrastructure, Transport and Tourism's Road Bridge Periodic Inspection Guidelines.

[0054] The deformation feature extraction unit 230 stores the function d(x', y') and the function L(x', y') obtained as described above in the deformation feature database 130 as deformation features.

[0055] The deformation characteristics stored in the deformation characteristics database 130 may be defined by a mathematical formula based on some kind of geometric structure, instead of being generated based on existing point cloud data containing deformations by the deformation characteristic data generation device 200. For example, in the case of a model based on an ellipsoid, the function d(x', y') may be defined by the following formula (3):

[0056]

number

[0057] The deformation feature synthesis unit 140 has a function of synthesizing the deformation features stored in the deformation feature database 130 in response to the input from the structural modeling unit 120. Figure 6 is an explanatory diagram showing an example of the synthesis of deformation features by the deformation feature synthesis unit 140.

[0058] The deformation feature synthesis unit 140 synthesizes the original point cloud data and the deformation feature by giving each point a displacement equivalent to the displacement amount d(x', y') shown in the speech bubble in Figure 6, which corresponds to the (x', y') coordinates when each point belonging to the selected range is projected onto a plane.

[0059] To synthesize the points, the deformation feature synthesizer 140 moves each point in the direction of the normal vector of the plane by an amount equivalent to the displacement d(x', y'). The normal vector of the plane is calculated, for example, using the following equation (4) based on the coefficients of the plane equation.

[0060]

number

[0061] The deformation feature synthesis unit 140 may also overwrite the label information of each point with the value of the function L(x', y') corresponding to the (x', y') coordinates when each point is projected onto a plane. The deformation synthesis data generation device 100 outputs point cloud data into which deformation features have been synthesized as described above.

[0062] As described above, the deformation feature synthesis unit 140 of this embodiment acquires the distribution of displacement amounts for points in the point cloud data and synthesizes the displacement amounts for points in the point cloud data according to the distribution. The distribution is the distribution of displacement amounts for each point in the point cloud data from a model that represents the shape formed by the point cloud data in an arbitrary range of the point cloud data.

[0063] The deformation synthesis range selection unit 110 of this embodiment selects an arbitrary range for synthesis in the point cloud data, and the structural modeling unit 120 generates a model for the arbitrary range selected in the point cloud data.

[0064] The model in this embodiment represents a planar structure. The structural modeling unit 120 generates the model by calculating the equation of a plane through which points belonging to an arbitrary range selected in the point cloud data pass.

[0065] The distribution of the displacement amounts is a distribution generated by calculating the displacement amounts from existing point cloud data. Note that the distribution of the displacement amounts may be a distribution defined by a mathematical formula that represents the relationship between the coordinates of points in the point cloud data and the displacement amounts.

[0066] Furthermore, the deformation characteristics database 130 of this embodiment stores the distribution of the amount of displacement.

[0067] [Explanation of operation] The operation of the deformation composite data generation device 100 of this embodiment will be described below with reference to Fig. 7. Fig. 7 is a flowchart showing the operation of the deformation feature synthesis process performed by the deformation composite data generation device 100 of the first embodiment.

[0068] First, the deformation characteristic data generation device 200 generates a function d(x', y') that represents the distribution of displacement as a deformation characteristic, and a function L(x', y') that represents label information. Next, the deformation characteristic data generation device 200 stores the generated deformation characteristics in the deformation characteristic database 130 (step S101).

[0069] Next, the deformation synthesis range selection unit 110 selects a range for deformation synthesis from the input point cloud data that is the target of deformation synthesis (step S102). Next, the deformation synthesis range selection unit 110 inputs the input original point cloud data and the selected range to the structural modeling unit 120.

[0070] Next, the structural modeling unit 120 models the selected range input from the deformation synthesis range selection unit 110 as a plane (step S103). Specifically, the structural modeling unit 120 calculates a plane equation through which points belonging to the selected range input from the deformation synthesis range selection unit 110 pass.

[0071] Next, the structural modeling unit 120 calculates the (x', y') coordinates of each point belonging to the selected range when it is projected onto the plane represented by the calculated plane equation. Next, the structural modeling unit 120 inputs the input original point cloud data, the calculated plane equation, and the (x', y') coordinates of each point belonging to the selected range to the deformation feature synthesis unit 140.

[0072] Next, the deformation feature synthesis unit 140 synthesizes the deformation features stored in the deformation feature database 130 in step S101 with the original point cloud data based on the input from the structural modeling unit 120 (step S104).

[0073] Next, the deformation feature synthesis unit 140 outputs the point cloud data into which the deformation feature has been synthesized (step S105). After outputting, the synthesized deformation data generation device 100 ends the deformation feature synthesis process.

[0074] [Effect description] The deformation composite data generation device 100 of this embodiment includes a deformation synthesis range selection unit 110 that selects the range for deformation synthesis for the input point cloud data, and a structural modeling unit 120 that calculates the planar structure including points that belong to the selected range in the input point cloud data.

[0075] In addition, the deformation composite data generation device 100 of this embodiment is equipped with a deformation feature database 130 that stores the deformation features to be synthesized, and a deformation feature synthesis unit 140 that synthesizes deformation features with respect to input point cloud data.

[0076] The composite deformation data generation device 100 of this embodiment focuses on the distribution of displacement from a plane as a deformation feature. In addition, by using a method of moving each point so that the deformation feature is reproduced in other point cloud data, the deformation feature synthesis unit 140 can generate point cloud data in which deformations including partial deformation of an object are synthesized.

[0077] As described above, the deformation synthetic data generation device 100 of this embodiment can generate point cloud data containing deformations. When the generated point cloud data is used, it is expected that the diversity of the training data will increase. Diversifying the training data will contribute to improving the detection performance of the constructed training model.

[0078] In addition, the deformation synthetic data generation device 100 of this embodiment can reduce the amount of actual data required to construct a learning model with a specified detection performance, thereby reducing the cost of data collection.

[0079] [Second embodiment] [Configuration Description] Next, a second embodiment of the present invention will be described with reference to the drawings. Fig. 8 is a block diagram showing an example of the configuration of a synthetic deformation data generating device according to the second embodiment of the present invention.

[0080] The deformation composite data generation device 101 shown in FIG. 8 includes a deformation composite range selection unit 110, a structural modeling unit 120, a deformation feature database 130, a deformation feature synthesis unit 140, and a deformation feature transformation unit 150.

[0081] 8, the deformation characteristic database 130 is communicably connected to the deformation characteristic data generation device 200. Note that the deformation characteristic database 130 does not have to be connected to the deformation characteristic data generation device 200.

[0082] The functions of the deformation synthesis range selection unit 110, the structural modeling unit 120, the deformation feature database 130, and the deformation feature synthesis unit 140 of this embodiment are similar to the functions of the first embodiment.

[0083] The deformation feature transformation unit 150 has the function of receiving the deformation features stored in the deformation feature database 130 and performing transformation processing such as enlarging or reducing the input deformation features. The deformation feature transformation unit 150 inputs the deformation features that have been transformed to the deformation feature synthesis unit 140.

[0084] The deformation feature transformation unit 150 performs enlargement or reduction, which is an example of a transformation process, as follows. When enlargement or reduction is performed parallel to the planar direction, the deformation feature transformation unit 150 sets the scale parameters in the x' and y' directions as s x ,s y Let's say.

[0085] Next, the deformation characteristic transforming unit 150 calculates the function d(x', y') representing the distribution of the amount of displacement as d1(x', y')=d(x' / s x ,y' / s y ) When performing the transformation, the deformation feature transformation unit 150 is required to apply the same transformation to the function L(x', y') representing the label information.

[0086] When enlarging or reducing the image in a direction perpendicular to the plane direction, the deformation characteristic transforming unit 150 sets the scale parameter to s z Then, the function d(x',y') is d2(x',y')=s z Transform it as d(x',y').

[0087] In addition to enlarging or reducing, the deformation feature transformer 150 can also perform transformations such as rotation and inversion. Furthermore, by limiting the domain of the function or substituting d(x', y') = 0 outside the range of a predetermined domain, the deformation feature transformer 150 can extract only a part of the deformation.

[0088] In addition, as a deformation process other than the above, the deformation characteristic deformation unit 150 may perform a deformation process that changes the shape of the deformation by multiplying or adding some function f(x',y'), such as d3(x',y')=f(x',y')d(x',y') or d4(x',y')=d(x',y')+f(x',y').

[0089] The function f(x',y') may be generated based on, for example, Perlin noise, which is a smoothly changing random number. The function f(x',y') may also be generated without using random numbers. The function f(x',y') is not limited to a specific function, and may be any function that affects the function d(x',y') that represents the distribution. The deformation characteristic transformation unit 150 can also combine the characteristics of two or more deformations by using a displacement extracted from another deformation as the function f(x',y').

[0090] The deformation characteristic deformation unit 150 may use a combination of the above-described deformation processes. The deformation process is not limited to a specific process as long as it affects the function d(x', y') that represents the distribution.

[0091] The deformation feature transformation unit 150 defines a function representing the distribution of the amount of displacement after the transformation process as function d(x',y'), and a function representing the label information after the transformation process as function L(x',y'). Next, the deformation feature transformation unit 150 inputs the functions d(x',y') and L(x',y') to the deformation feature synthesis unit 140.

[0092] As described above, the deformation feature deformation unit 150 of this embodiment performs deformation processing on the distribution of the amount of displacement.

[0093] [Explanation of operation] The operation of the deformation composite data generation device 101 of this embodiment will be described below with reference to Fig. 9. Fig. 9 is a flowchart showing the operation of the deformation feature synthesis process performed by the deformation composite data generation device 101 of the second embodiment.

[0094] The processes in steps S201 to S203 are similar to the processes in steps S101 to S103 shown in FIG.

[0095] Next, the deformation feature transformation unit 150 receives the deformation features stored in the deformation feature database 130 as input and performs a transformation process on the input deformation features (step S204). The deformation feature transformation unit 150 inputs the transformed deformation features to the deformation feature synthesis unit 140.

[0096] Next, the deformation feature synthesis unit 140 synthesizes the deformation features input from the deformation feature transformation unit 150 in step S204 with the original point cloud data based on the input from the structural modeling unit 120 (step S205). The processing in step S206 is the same as the processing in step S105 shown in FIG.

[0097] [Effect description] The composite deformation data generation device 101 of this embodiment is equipped with a deformation feature transformation unit 150 that performs transformation processes such as enlargement and reduction on the deformation features. Because the deformation feature transformation unit 150 can perform various transformation processes on the deformation features, the composite deformation data generation device 101 can output a variety of point cloud data into which deformation features have been combined. In other words, the composite deformation data generation device 101 can further increase the diversity of the training data.

[0098] [Third embodiment] [Configuration Description] Next, a third embodiment of the present invention will be described with reference to the drawings. Fig. 10 is a block diagram showing an example of the configuration of a deformation composite data generating device according to the third embodiment of the present invention.

[0099] The deformation composite data generation device 102 shown in FIG. 10 includes a deformation composite range selection unit 111, a structural modeling unit 121, a deformation feature database 131, and a deformation feature synthesis unit 141.

[0100] 10, the deformation characteristic database 131 is communicably connected to the deformation characteristic data generation device 201. Note that the deformation characteristic database 131 does not have to be connected to the deformation characteristic data generation device 201.

[0101] The deformation composite data generation device 102 of this embodiment is characterized by the fact that it is not limited to areas with planar structures as in the first embodiment, but also synthesizes deformations for areas with other geometric structures such as cylinders (especially side surfaces) and spheres.

[0102] The functions of the deformation synthesis range selection unit 111, structural modeling unit 121, deformation feature database 131, and deformation feature synthesis unit 141 are similar to the functions of the deformation synthesis range selection unit 110, structural modeling unit 120, deformation feature database 130, and deformation feature synthesis unit 140 in the first embodiment, except that they are not limited to planar structures.

[0103] Furthermore, the functions of the deformation characteristic data generation device 201 shown in FIG. 10 are the same as the functions of the deformation characteristic data generation device 200 in the first embodiment, except that it is not limited to planar structures.

[0104] The differences between this embodiment and the first embodiment will be explained below. The deformation synthesis range selection unit 111 may select a range for selecting a region having a geometric structure other than a region having a planar structure, such as a cylinder or a sphere. The deformation synthesis range selection unit 111 selects a range for deformation synthesis for a geometric structure that has been manually identified using point cloud processing software, for example.

[0105] Alternatively, the deformation synthesis range selection unit 111 may select a range for performing deformation synthesis for the geometric structure extracted using an algorithm based on RANSAC described in Non-Patent Document 3.

[0106] The structural modeling unit 121 has a function of modeling the selected range input from the deformation synthesis range selection unit 110 as a geometric structure. Specifically, it calculates an equation through which points belonging to the selected range input from the deformation synthesis range selection unit 111 pass, or calculates parameters that characterize the geometric structure.

[0107] When the geometric structure is a plane, the structural modeling unit 121 calculates an equation in the form of Equation (1). When the geometric structure is a spherical surface, the structural modeling unit 121 calculates an equation in the form of Equation (5).

[0108]

number

[0109] Fig. 11 is an explanatory diagram showing an example of a cylinder that is treated as a geometric structure by the deformation composite data generation device 102. When the geometric structure is a cylinder, the structural modeling unit 121 can characterize the cylinder using a set of parameters, namely, the position and direction of the central axis and the radius r, as shown in Fig. 11.

[0110] After calculating the equations or parameters, the structural modeling unit 121 determines a coordinate system on the geometric structure. In the first embodiment, the Cartesian coordinate system corresponds to the coordinate system on the geometric structure (plane).

[0111] When the geometric structure is not a plane but a cylinder, each point on the cylinder is expressed by a combination of a position u on the central axis and an angle θ as shown in Fig. 11. That is, the structural modeling unit 121 can use the (u, θ) coordinates.

[0112] Alternatively, the structural modeling unit 121 can use (u, l) coordinates in which the arc length l = rθ is used instead of the angle. The (u, l) coordinates are considered to be more suitable for expressing information such as the magnitude of deformation.

[0113] After determining the coordinate system on the geometric structure, the structural modeling unit 121 calculates the coordinates of each point belonging to the selected range when the point is projected onto the geometric structure as shown in Fig. 12. Fig. 12 is an explanatory diagram showing an example of the projection of points belonging to the selected range onto the geometric structure by the structural modeling unit 121.

[0114] The deformation characteristics stored in the deformation characteristics database 131 of this embodiment are functions in a coordinate system on the geometric structure that express the amount of displacement from the geometric structure.

[0115] Furthermore, the deformation feature extraction unit 230 of this embodiment can define the amount of displacement from the geometric structure as, for example, the Euclidean distance between a point after being projected onto the geometric structure and the original point. d shown in FIG. 12 corresponds to the Euclidean distance. Note that the deformation feature extraction unit 230 may also define the amount of displacement from the geometric structure as a signed distance with a negative sign pointing inward and a positive sign pointing outward.

[0116] Also, similar to the first embodiment, when each point has label information that is information such as the presence or absence of deformation, the type of deformation, or the scale of deformation, the deformation feature extraction unit 230 also defines a function that represents the label information.

[0117] When the (u, l) coordinate system is used, the deformation feature extraction unit 230 stores the function d(u, l) representing the distribution of displacement and the function L(u, l) representing label information as deformation features in the deformation feature database 131. The deformation feature data generation device 201, which does not limit its functionality to planar structures, can generate the above-mentioned deformation features from existing point cloud data containing deformations.

[0118] When the (u, l) coordinate system is used, the deformation feature synthesis unit 141 synthesizes the original point cloud data and the deformation feature by giving each point belonging to the selected range a displacement equivalent to the displacement amount d(u, l) corresponding to the (u, l) coordinate when the point is projected onto the geometric structure.

[0119] To synthesize the points, the deformation feature synthesizing unit 141 moves each point by an amount corresponding to the displacement d(u, l) in the direction of the normal vector of the geometric structure. Note that, unlike the normal vector of a plane represented by Equation (4), the normal vector of a general geometric structure depends on the position.

[0120] For example, in the case of a cylinder whose central axis is the same as the z-axis, the normal vector is calculated as (cosθ, sinθ, 0). Note that if the central axis is different from the z-axis, the normal vector will rotate by the same amount.

[0121] As described above, the model of this embodiment represents a geometric structure such as a sphere or a cylinder. The structural modeling unit 121 of this embodiment generates a model by calculating an equation of a sphere or a cylinder through which points belonging to an arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure.

[0122] [Explanation of operation] The operation of the deformation composite data generation device 102 of this embodiment will be described below with reference to Fig. 13. Fig. 13 is a flowchart showing the operation of the deformation feature synthesis process by the deformation composite data generation device 102 of the third embodiment.

[0123] The processes in steps S301 to S304 are similar to the processes in steps S101 to S104 shown in Fig. 7, except that they are not limited to a planar structure. Also, the process in step S305 is similar to the process in step S105 shown in Fig. 7.

[0124] [Effect description] The composite deformation data generation device 102 of this embodiment does not limit the objects for which deformations are to be synthesized to parts with planar structures, and can therefore synthesize deformations for a greater variety of parts than in the first embodiment. For example, the composite deformation data generation device 102 of this embodiment can also synthesize deformations for parts such as the piers of a bridge that are cylindrical in shape.

[0125] [Fourth embodiment] [Configuration Description] Next, a fourth embodiment of the present invention will be described with reference to the drawings. Fig. 14 is a block diagram showing an example of the configuration of a deformation composite data generating device according to the fourth embodiment of the present invention.

[0126] The deformation composite data generation device 103 shown in FIG. 14 includes a deformation composite range selection unit 110, a structural modeling unit 120, a deformation feature database 130, a deformation feature synthesis unit 140, and a point cloud learning unit 160.

[0127] 14, the deformation characteristic database 130 is communicably connected to the deformation characteristic data generation device 200. Note that the deformation characteristic database 130 does not have to be connected to the deformation characteristic data generation device 200.

[0128] The functions of the deformation synthesis range selection unit 110, structural modeling unit 120, deformation feature database 130, and deformation feature synthesis unit 140 of this embodiment are the same as those of the first embodiment. Note that it is preferable that the point cloud data output by the deformation feature synthesis unit 140 is not just one, but multiple point cloud data in which deformations of various shapes are synthesized.

[0129] The point cloud learning unit 160 has the function of performing machine learning using the point cloud data output by the deformation feature synthesis unit 140 as learning data, and constructing a machine learning model or deep learning model that determines whether or not there is a deformation, or identifies the location of the deformation, etc.

[0130] In addition, instead of using only the synthesized data output by the deformation feature synthesis unit 140 as training data, the point cloud learning unit 160 may use data that is a mixture of data that originally has a deformation and synthesized data as training data.

[0131] The point cloud learning unit 160 constructs, for example, a machine learning network or a deep learning network as a machine learning model or a deep learning model. The machine learning network or deep learning network that uses point cloud data as input is, for example, PointNet++ described in Non-Patent Document 4.

[0132] After completing the learning, the point cloud learning unit 160 outputs the constructed machine learning model or deep learning model. The learning model output by the point cloud learning unit 160 is a learning model for detecting anomalies that can detect anomalies.

[0133] As described above, the point cloud learning unit 160 of this embodiment constructs a learning model by performing machine learning using the point cloud data into which the displacement amounts have been combined as learning data.

[0134] [Explanation of operation] The operation of the deformation composite data generation device 103 of this embodiment will be described below with reference to Fig. 15. Fig. 15 is a flowchart showing the operation of the deformation feature synthesis process by the deformation composite data generation device 103 of the fourth embodiment.

[0135] The processes in steps S401 to S404 are similar to the processes in steps S101 to S104 shown in FIG.

[0136] The deformation feature synthesis unit 140 checks whether the number of point cloud data into which the deformation features have been synthesized has reached a predetermined number (step S405). If the number of point cloud data into which the deformation features have been synthesized has not reached the predetermined number (No in step S405), the synthesized deformation data generation device 103 repeatedly executes the processes of steps S401 to S404.

[0137] By repeatedly executing the processes of steps S401 to S404, the deformation composite data generation device 103 can generate a plurality of point cloud data in which deformations of various shapes are combined. Note that the process of step S401 may be omitted as appropriate.

[0138] When the number of point cloud data into which deformation features have been combined reaches a predetermined number (Yes in step S405), the deformation feature combination unit 140 inputs the plurality of point cloud data into which deformation features have been combined to the point cloud learning unit 160. The point cloud learning unit 160 constructs a machine learning model or a deep learning model using the point cloud data input from the deformation feature combination unit 140 as learning data (step S406).

[0139] Next, the point cloud learning unit 160 outputs the machine learning model or deep learning model constructed in step S406 (step S407). After outputting, the deformation composite data generation device 103 ends the deformation feature synthesis process.

[0140] [Effect description] The composite deformation data generation device 103 of this embodiment includes a point cloud learning unit 160 that constructs a learning model for detecting deformations using the point cloud data output by the deformation feature synthesis unit 140 as learning data. By using multiple point cloud data in which various deformations are synthesized as learning data, the point cloud learning unit 160 can improve the performance of machine learning models and deep learning models that determine the presence or absence of deformations or identify the location of deformations.

[0141] A specific example of the hardware configuration of the deformation combined data generation devices 100 to 103 of each embodiment will be described below. Fig. 16 is an explanatory diagram showing an example of the hardware configuration of the deformation combined data generation device according to the present invention.

[0142] The deformation composite data generation device shown in Fig. 16 includes a CPU (Central Processing Unit) 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. It also includes an input unit 15 for user operation and an output unit 16 for presenting the processing results or the progress of the processing contents to the user.

[0143] The deformation composite data generating device is realized by software when the CPU 11 shown in FIG. 16 executes a program that provides the functions of each component.

[0144] That is, the CPU 11 loads the program stored in the auxiliary storage unit 14 into the main storage unit 12, executes it, and controls the operation of the deformation composite data generation device, thereby realizing each function by software.

[0145] The deformation composite data generation device shown in Fig. 16 may include a DSP (Digital Signal Processor) instead of the CPU 11. Alternatively, the deformation composite data generation device shown in Fig. 16 may include both the CPU 11 and a DSP.

[0146] The main memory unit 12 is used as a data working area and a data temporary saving area. The main memory unit 12 is, for example, a RAM (Random Access Memory). Deformation characteristic databases 130 to 131 are realized by the main memory unit 12.

[0147] The communication unit 13 has a function of inputting and outputting data to and from peripheral devices via a wired network or a wireless network (information communication network).

[0148] The auxiliary storage unit 14 is a non-transitory tangible storage medium, such as a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a semiconductor memory.

[0149] The input unit 15 has a function of inputting data and processing commands, and is an input device such as a keyboard or a mouse.

[0150] The output unit 16 has a function of outputting data and is, for example, a display device such as a liquid crystal display device, or a printing device such as a printer.

[0151] As shown in FIG. 16, in the deformation composite data generation device, each component is connected to a system bus 17.

[0152] In the deformation composite data generation device 100 of the first embodiment, the auxiliary storage unit 14 stores programs for realizing the deformation composite range selection unit 110, the structural modeling unit 120, and the deformation feature composite unit 140.

[0153] The deformation composite data generation device 100 may be implemented with a circuit including hardware components such as an LSI (Large Scale Integration) that realizes the functions shown in FIG. 1 inside.

[0154] In addition, in the second embodiment of the deformation composite data generation device 101, the auxiliary memory unit 14 stores programs for realizing the deformation composite range selection unit 110, the structural modeling unit 120, the deformation feature synthesis unit 140, and the deformation feature transformation unit 150.

[0155] The deformation composite data generation device 101 may be implemented with a circuit including hardware components such as an LSI that realizes the functions shown in FIG. 8, for example.

[0156] In addition, in the deformation composite data generation device 102 of the third embodiment, the auxiliary memory unit 14 stores programs for realizing the deformation composite range selection unit 111, the structural modeling unit 121, and the deformation feature composite unit 141.

[0157] The deformation composite data generation device 102 may be implemented with a circuit including hardware components such as an LSI that realizes the functions shown in FIG. 10, for example.

[0158] In addition, in the fourth embodiment of the deformation composite data generation device 103, the auxiliary memory unit 14 stores programs for realizing the deformation synthesis range selection unit 110, the structural modeling unit 120, the deformation feature synthesis unit 140, and the point cloud learning unit 160.

[0159] The deformation composite data generation device 103 may be implemented with a circuit including hardware components such as an LSI that realizes the functions shown in FIG. 14, for example.

[0160] Furthermore, the deformation composite data generation devices 100-103 may be realized by hardware that does not include computer functions using elements such as a CPU. For example, some or all of the components may be realized by general-purpose circuits, dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip (for example, the above-mentioned LSI), or by multiple chips connected via a bus. Some or all of the components may be realized by a combination of the above-mentioned circuits, etc., and a program.

[0161] Furthermore, some or all of the components of the deformation composite data generation devices 100 to 103 may be configured by one or more information processing devices each having a calculation unit and a storage unit.

[0162] When some or all of the components are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in which they are connected via a communication network.

[0163] Next, an overview of the present invention will be described. Figure 17 is a block diagram showing an overview of a synthetic deformation data generation device according to the present invention. A synthetic deformation data generation device 20 according to the present invention includes an acquisition unit 21 (e.g., a deformation feature synthesis unit 140) that acquires the distribution of displacement amounts for points in point cloud data, and a synthesis unit 22 (e.g., a deformation feature synthesis unit 140) that synthesizes the displacement amounts according to the distribution for points in the point cloud data.

[0164] The distribution is a distribution of displacements of points in the point cloud data from a model that represents the shape formed by the point cloud data in an arbitrary range of the point cloud data. The distribution of displacements may be a distribution generated by calculating displacements from existing point cloud data. The distribution of displacements may also be a distribution defined by a mathematical formula that represents the relationship between the coordinates of points in the point cloud data and the displacements.

[0165] With such a configuration, the deformation composite data generation device can generate new data containing deformations even if the data format is a point cloud.

[0166] The deformation composite data generating device 20 may further include a selection unit (e.g., a deformation synthesis range selection unit 110) that selects an arbitrary range for synthesis in the point cloud data. The deformation composite data generating device 20 may further include a generation unit (e.g., a structural modeling unit 120) that generates a model for an arbitrary range selected in the point cloud data.

[0167] With such a configuration, the deformation composite data generation device can generate new data having deformations based on any range in the point cloud data.

[0168] The model may represent a planar structure, and the generator may generate the model by calculating an equation of a plane through which points belonging to an arbitrary range selected in the point cloud data pass.

[0169] With such a configuration, the deformation composite data generation device can generate new data having deformations based on point cloud data representing a plane.

[0170] The model may also represent a spherical or cylindrical geometric structure, and the generator may generate the model by calculating an equation of the sphere or cylinder through which points belonging to an arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure.

[0171] With such a configuration, the deformation synthetic data generation device can generate new data having deformations based on point cloud data representing a sphere or a cylinder.

[0172] The deformation composite data generation device 20 may further include a deformation unit (for example, a deformation feature deformation unit 150) that performs a deformation process on the distribution of the amount of displacement.

[0173] With such a configuration, the deformation synthetic data generation device can further increase the diversity of the training data.

[0174] The deformation composite data generation device 20 may further include a learning unit (for example, a point cloud learning unit 160) that constructs a learning model by performing machine learning using the point cloud data into which the displacement amount has been combined as learning data.

[0175] With such a configuration, the deformation synthetic data generation device can construct a machine learning model or deep learning model that determines whether or not there is a deformation, or identifies the location of the deformation.

[0176] The deformation composite data generation device 20 may further include a storage unit (for example, a deformation characteristics database 130) that stores the distribution of the amount of displacement.

[0177] Furthermore, some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0178] (Appendix 1) A deformation composite data generation device characterized by comprising an acquisition unit that acquires a distribution of displacement amounts for points of point cloud data, and a synthesis unit that synthesizes the displacement amounts for the points in the point cloud data according to the distribution.

[0179] (Appendix 2) A deformation synthetic data generation device as described in Appendix 1, wherein the distribution is a distribution of the amount of displacement of each point in the point cloud data from a model representing the shape formed by the point cloud data in any range of the point cloud data.

[0180] (Appendix 3) The deformation composite data generation device according to Appendix 2, further comprising a selection unit for selecting the arbitrary range to be composited for the point cloud data.

[0181] (Appendix 4) The deformation synthetic data generation device according to appendix 3, further comprising a generation unit that generates the model for the arbitrary range selected in the point cloud data.

[0182] (Appendix 5) A deformation synthetic data generation device as described in Appendix 4, wherein the model represents a planar structure, and the generation unit generates the model by calculating an equation of a plane through which points belonging to the arbitrary range selected in the point cloud data pass.

[0183] (Appendix 6) The model represents a spherical or cylindrical geometric structure, and the generation unit generates the model by calculating an equation of a sphere or a cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure. A deformation synthetic data generation device as described in Appendix 4.

[0184] (Supplementary Note 7) The deformation composite data generation device according to any one of Supplementary Note 1 to Supplementary Note 6, further comprising a deformation unit that performs deformation processing on the distribution of the amount of displacement.

[0185] (Appendix 8) A deformation composite data generation device described in any one of Appendices 1 to 7, further comprising a learning unit that constructs a learning model by performing machine learning using the point cloud data into which the displacement amount has been synthesized as learning data.

[0186] (Supplementary Note 9) The deformation composite data generation device according to any one of Supplementary Note 1 to Supplementary Note 8, further comprising a storage unit that stores the distribution of the amount of displacement.

[0187] (Appendix 10) A deformation composite data generation device according to any one of Appendices 1 to 9, wherein the distribution of displacement amounts is a distribution generated by calculating displacement amounts from existing point cloud data.

[0188] (Appendix 11) A deformation synthetic data generation device described in any one of Appendices 1 to 9, wherein the distribution of displacement is a distribution defined by a mathematical formula representing the relationship between the coordinates of points in the point cloud data and the displacement amount.

[0189] (Appendix 12) A deformation composite data generation method characterized by obtaining a distribution of displacement amounts for points of point cloud data and synthesizing the displacement amounts for the points in the point cloud data according to the distribution.

[0190] (Appendix 13) A computer-readable recording medium having recorded thereon a deformation composite data generation program that acquires a distribution of displacement amounts for points of point cloud data and synthesizes the displacement amounts for the points in the point cloud data according to the distribution.

[0191] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0192] The present invention is suitable for use in developing machine learning models and deep learning models that use point cloud data to detect damage such as cracks, peeling, and exposed rebar in infrastructure facilities such as bridges, tunnels, and concrete structures. [Explanation of symbols]

[0193] 11 CPU 12 Main memory 13 Communications Department 14 Auxiliary storage 15 Input section 16 Output section 17 System Bus 20, 100-103 Deformation synthetic data generator 21 Acquisition Department 22 Synthesis section 110, 111 Deformation synthesis range selection section 120, 121, 220 Structural Modeling Division 130, 131 Deformation characteristics database 140, 141 Deformation feature synthesis section 150 Deformation characteristics Deformed part 160 Point Cloud Learning Unit 200, 201 Deformation characteristic data generation device 210 Deformation extraction range selection section 230 Deformation feature extraction unit

Claims

1. an acquisition unit that acquires a distribution of displacement amounts for points of the point cloud data; a synthesis unit that synthesizes the displacement amounts to the points in the point cloud data according to the distribution, the distribution is a distribution of displacement amounts of each point in the point cloud data from a model representing a shape formed by the point cloud data in an arbitrary range of the point cloud data, a selection unit that selects the arbitrary range to be synthesized with respect to the point cloud data; a generation unit that generates the model for the arbitrary range selected in the point cloud data, the model represents a spherical or cylindrical geometry; The generation unit generates the model by calculating an equation of a sphere or a cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure. A deformation synthetic data generation device characterized by:

2. Further provided is a deformation unit that performs deformation processing on the distribution of the displacement amount. The deformation synthetic data generating device according to claim 1.

3. The point cloud data with the displacement amounts synthesized therein is used as learning data to perform machine learning to construct a learning model. The deformation synthetic data generating device according to claim 1.

4. Obtain the distribution of displacement for points in the point cloud data, synthesizing the displacement amounts to the points in the point cloud data according to the distribution; the distribution is a distribution of displacement amounts of each point in the point cloud data from a model representing a shape formed by the point cloud data in an arbitrary range of the point cloud data, Select the arbitrary range to be synthesized with respect to the point cloud data, generating the model for the selected arbitrary range in the point cloud data; the model represents a spherical or cylindrical geometry; When generating the model, the model is generated by calculating an equation of a sphere or a cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure. A deformation synthetic data generation method characterized by:

5. On the computer, A process of acquiring a distribution of displacement amounts for points of the point cloud data; and executing a process of synthesizing the displacement amounts at the points in the point cloud data in accordance with the distribution; the distribution is a distribution of displacement amounts of each point in the point cloud data from a model representing a shape formed by the point cloud data in an arbitrary range of the point cloud data, The computer, A process of selecting the arbitrary range to be synthesized with respect to the point cloud data; and generating the model for the arbitrary range selected in the point cloud data; the model represents a spherical or cylindrical geometry; The computer, When generating the model, the model is generated by calculating an equation of a sphere or a cylinder through which points belonging to the arbitrary range selected in the point cloud data pass, or parameters that characterize the geometric structure. Deformation synthetic data generation program for

Images