3D model generation method, generation system, and program

Abstract

To provide a system, a method, and a program for generating a three-dimensional model.SOLUTION: A method for generating a three-dimensional model includes: (1) deriving an initial reference coordinate system based on measurement data indicating a three-dimensional shape of an object; (2) selecting an axis of the initial reference coordinate system; (3) determining the presence of rotational symmetry about the axis; (4) determining the presence of rotational symmetry with an angular period about the axis; (5) in the presence of the rotational symmetry with the angular period, correcting a reference coordinate system; (6) in the absence of rotational symmetry, determining plane symmetry about the axis; (7) in the presence of the plane symmetry, correcting the reference coordinate system; and (8) performing the processes (3)-(7) for each of three axes of the initial reference coordinate system.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a method, a system, and a program for generating a three-dimensional model. [Background technology]

[0002] Patent Document 1 discloses an information processing device that calculates the position and orientation of an object. The information processing device of Patent Document 1 acquires a three-dimensional shape model (edge ​​information, point cloud information) from distance point cloud information of the object, and determines rotational symmetry around a first axis. The information processing device then determines inversion symmetry around a second axis different from the first axis. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-85781 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, rotational symmetry around the first axis and inversion symmetry around the second axis are determined. Therefore, the coordinate axes must be set by a human, and it is difficult to set the coordinate axes automatically. Therefore, it is difficult to efficiently generate 3D models of various objects. In this way, a 3D model can be created in a reference coordinate system that takes into account the symmetry of the object.

[0005] In generating a 3D shape model, for example, by setting coordinate axes that take into account the symmetry of an object, it is possible to annotate appropriate information. Therefore, it is possible to generate learning data to which position and orientation information regarding the position and orientation of an object is added. It is possible to generate an object estimator by supervised learning using the object position and orientation information as the correct answer label (teaching data).

[0006] The present disclosure has been made to solve such problems, and provides a 3D model generation method, generation system, and program that can efficiently generate a 3D model in an appropriate coordinate system. [Means for solving the problem]

[0007] In this embodiment, the method for generating a three-dimensional model includes at least one processor: (1) deriving an initial reference coordinate system based on measurement data indicating the three-dimensional shape of an object; (2) selecting an axis of the initial reference coordinate system; (3) determining whether or not there is rotational symmetry about the axis; (4) determining whether or not there is rotational symmetry with an angular period about the axis; (5) if there is no rotational symmetry, determining whether or not there is planar symmetry about the axis; and (6) changing the axis selected in (2), thereby performing processes (3) to (5) for each of the three axes of the initial reference coordinate system.

[0008] In the above method, when there is rotational symmetry having the angular period, or when there is planar symmetry, the initial reference coordinate system may be modified.

[0009] In the above method, the initial reference coordinate system may be derived based on a direction of a normal to each mesh of a mesh model representing the three-dimensional shape.

[0010] In the above method, a rectangular parallelepiped containing the object may be used as a three-dimensional bounding box, and an initial reference coordinate system may be derived in which the center of the rectangular parallelepiped is the origin and each side direction of the rectangular parallelepiped is the x, y and z axes.

[0011] In the above method, when the plane symmetry is present, the axial directions of the initial reference coordinate system may be corrected so as to have axes along the inverted plane.

[0012] In the above method, when the object has rotational symmetry with the angular period, the axial directions of the initial reference coordinate system may be corrected based on the peak surface.

[0013] The system in this embodiment is a 3D model generation system having at least one processor, in which the processor (1) derives an initial reference coordinate system based on measurement data indicating the 3D shape of an object, (2) selects an axis of the initial reference coordinate system, (3) determines whether there is rotational symmetry about the axis, (4) determines whether there is rotational symmetry with an angular period about the axis, (5) if there is no rotational symmetry, determines planar symmetry about the axis, and (6) performs processes (3) to (5) for each of the three axes of the initial reference coordinate system by changing the axis selected in (2).

[0014] In the above system, when there is rotational symmetry having the angular period, or when there is plane symmetry, the initial reference coordinate system may be corrected.

[0015] In the above system, the initial reference coordinate system may be derived based on a direction of a normal to each mesh of a mesh model representing the three-dimensional shape.

[0016] In the above system, a rectangular parallelepiped containing the object may be defined as a three-dimensional bounding box, and an initial reference coordinate system may be derived in which the center of the rectangular parallelepiped is defined as the origin and each side direction of the rectangular parallelepiped is defined as an xyz axis.

[0017] In the above system, when the system has the plane symmetry, the axial directions of the initial reference coordinate system may be corrected so as to have axes along the inverted plane.

[0018] In the above system, when the system has rotational symmetry with the angular period, the axial directions of the initial reference coordinate system may be corrected based on the peak surface.

[0019] The program in this embodiment is a program for causing a computer to execute a method for generating a three-dimensional model, the method including the steps of (1) deriving an initial reference coordinate system based on measurement data indicating the three-dimensional shape of an object, (2) selecting an axis of the initial reference coordinate system, (3) determining whether or not there is rotational symmetry with respect to the axis, (4) determining whether or not there is rotational symmetry with an angular period with respect to the axis, (5) if there is no rotational symmetry, determining whether or not there is planar symmetry with respect to the axis, and (6) changing the axis selected in (2) to perform the processes in (3) to (5) for each of the three axes of the initial reference coordinate system.

[0020] The present disclosure makes it possible to provide a 3D model generating method, generation system, and program that can efficiently generate a 3D model in an appropriate coordinate system. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram showing a system configuration. [Diagram 2] 4 is a flowchart showing a method for generating a three-dimensional model according to the present embodiment. [Diagram 3] 13 is a flowchart showing a process of setting an initial reference coordinate system. [Figure 4] FIG. 1 is a diagram for explaining a three-dimensional bounding box. [Diagram 5] 13 is a flowchart showing a process for determining rotational symmetry. [Figure 6] 1 is a diagram for explaining a cross-sectional shape 1c and r(θ) of a convex hull of an object. [Figure 7] 1 is a graph showing r'(θ) and its Fourier transform. [Figure 8] 13 is a graph showing the difference between r(θ) and r(θ+ωmax). [Figure 9] 13 is a flowchart showing a process of determining plane symmetry. [Figure 10]13 is a flowchart showing a process of determining plane symmetry. [Figure 11] 13A to 13C are diagrams showing cross-sectional shapes in a process of determining plane symmetry. [Figure 12] 13A and 13B are diagrams for explaining a process of correcting an axis in a process of determining plane symmetry. [Figure 13] 2 is a block diagram showing an example of a hardware configuration of a processing device 100. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems.

[0023] In recent years, machine learning models have been used as estimators for estimating the position and orientation of objects. In order to generate such estimators using machine learning, it is necessary to collect a large amount of training data. The training data may include not only images captured by a camera, but also synthetic images generated by simulation.

[0024] A renderer or simulator generates a synthetic image from a three-dimensional model (also called a three-dimensional shape model or 3D model) that shows the three-dimensional shape of an object. By using a simulator or similar, it is possible to easily generate a large number of synthetic images in which the position and orientation of an object are changed. Furthermore, a simulator or similar can annotate labels such as position and orientation on the synthetic images.

[0025] To create a 3D model, sensors such as a stereo camera and a depth sensor are used. For example, a depth sensor measures point cloud data of an object. The point cloud data includes information on the position coordinates of each point in an XYZ three-dimensional orthogonal coordinate system. In addition, a 3D model can be formed by converting the point cloud data into edge data, mesh data, surface data, or the like.

[0026] Here, in a three-dimensional model showing the three-dimensional shape of an object, it is desirable to appropriately set the coordinate axes of the coordinate system. For example, if an object is rotationally symmetric, the position and orientation of the object can be appropriately changed by setting coordinate axes that coincide with the rotation axes. It is desirable to appropriately determine the symmetry of an object. In the method according to the present embodiment, the symmetry of an object can be appropriately determined. In this way, it is possible to set a coordinate system that is useful for estimating the six-axis position and orientation in an estimator.

[0027] A three-dimensional model generating system and a generating method according to the present embodiment will be described with reference to the drawings. Fig. 1 is a block diagram showing a configuration of a system 1. The system 1 includes a processing device 100 and a sensor 200. The processing device 100 performs a process for determining the symmetry of an object O.

[0028] The sensor 200 is a measuring instrument for measuring the three-dimensional shape of an object O. The sensor 200 is, for example, a three-dimensional scanner. The sensor 200 includes a table 210 and a depth sensor 220. The table 210 has a placement surface parallel to a horizontal plane. A user or a robot places an object O to be measured on the table 210. The depth sensor 220 measures the distance to the object O on the table 210. This makes it possible to capture a depth image. The table 210 rotates the object O around a rotation axis parallel to the vertical direction. Then, while the table 210 is rotating the object O, the depth sensor 220 measures the distance to the surface of the object O.

[0029] This allows the sensor 200 to measure measurement data indicating the three-dimensional shape of the entire circumference of the object O. The measurement data includes a plurality of distance measurement data indicating the distance from the depth sensor 220 to the object O. Since the object O can be measured while the table 210 is rotating, the distance to the surface of the object O from any direction can be measured. Furthermore, in order to measure the shape of the bottom surface or top surface of the object O, the sensor 200 may perform measurements by changing the orientation of the object O on the table 210.

[0030] The measurement data measured by the sensor 200 is, for example, point cloud data having a plurality of points. Each point includes information of three-dimensional coordinates in the sensor coordinate system. The measurement data may also be mesh data. The mesh data is configured of a triangular mesh (tri-mesh) connecting three adjacent points or a quadrangular mesh (quad-mesh) connecting four adjacent points. The sensor 200 outputs the measurement data to the processing device 100.

[0031] The processing device 100 is an information processing device of a personal computer. For example, the processing device 100 includes a memory, a processor, various interfaces, an input device, an output device, a monitor, and the like. The processor of the processing device 100 executes a program stored in the memory to perform processing described below. Furthermore, the processing device 100 is an information processing device capable of wireless or wired communication.

[0032] The processing device 100 includes a measurement data acquisition unit 110 , a coordinate system setting unit 120 , a first determination unit 130 , a second determination unit 140 , a third determination unit 150 , and a correction unit 160 .

[0033] The measurement data acquisition unit 110 acquires measurement data measured by the sensor 200. As described above, the measurement data is point cloud data or mesh data indicating the three-dimensional shape of an object. That is, the processing device 100 acquires data of a three-dimensional model indicating the three-dimensional shape (three-dimensional data). In the following, the measurement data acquisition unit 110 will be described as acquiring a mesh model indicating the three-dimensional shape. Of course, the measurement data acquisition unit 110 may acquire point cloud data from the sensor 200, and the processing device 100 may generate mesh data from the point cloud data.

[0034] The coordinate system setting unit 120 sets a reference coordinate system for the three-dimensional model represented by the measurement data. To this end, the coordinate system setting unit 120 first derives an initial reference coordinate system. An XYZ three-dimensional orthogonal coordinate system is set as the initial reference coordinate system. The coordinate system setting unit 120 sets an origin and directions of the X-axis, Y-axis, and Z-axis for the three-dimensional model. The X-axis, Y-axis, and Z-axis are mutually orthogonal directions.

[0035] The first determination unit 130 determines the rotational symmetry of the object O. The first determination unit 130 determines whether or not the object O is rotationally symmetric about each of the three axes. For example, when the object O has a rotationally symmetric shape about the X axis, the X axis becomes the axis of rotational symmetry. Therefore, when the object is rotated by a certain angle around the X axis, the shape of the object matches.

[0036] The second determination unit 140 determines whether the object O is rotationally symmetric with an angular period. When there is a rotationally symmetric axis, the second determination unit 140 determines whether the rotational symmetry about the axis has an angular period. For example, when the X-axis is the rotationally symmetric axis, the second determination unit 140 determines whether the object O has a rotationally symmetric shape with an angular period about the X-axis.

[0037] An example of a rotationally symmetric shape with angular period is a regular polygon in a cross section perpendicular to an axis. Specifically, when the cross section is a regular square, the object has a four-fold rotationally symmetric shape. An example of a rotationally symmetric shape without angular period is a circle in a cross section perpendicular to an axis.

[0038] The third determination unit 150 determines plane symmetry. If the object shape has plane symmetry, the shape is inversion symmetric with respect to a plane. If the shape is inversion symmetric with respect to a plane that includes an axis, the third determination unit 150 determines that the object is plane symmetric.

[0039] The modification unit 160 modifies the reference coordinate system. If the second determination unit 140 determines that the coordinate system is rotationally symmetric with an angular period, the modification unit 160 modifies the reference coordinate system. This sets a reference coordinate system different from the initial reference coordinate system. Specifically, the modification unit 160 modifies the orientation of an axis that is not a rotationally symmetric axis. For example, if the X-axis is a rotationally symmetric axis, the modification unit 160 changes the orientation of the Y-axis and the Z-axis. The modification unit 160 rotates the Y-axis and the Z-axis around the X-axis.

[0040] Furthermore, when the third determination unit 150 determines that the object O has plane symmetry, the modification unit 160 modifies the reference coordinate system. That is, the axis directions of the reference coordinate system are changed so that the axes are included in the symmetrical plane. In this way, it is possible to set a reference coordinate system whose coordinate axes are two axes parallel to the symmetrical plane.

[0041] A method for generating a three-dimensional model according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the method for generating a three-dimensional model according to this embodiment.

[0042] The measurement data acquisition unit 110 acquires measurement data measured by the sensor 200 (S101). The measurement data is, for example, point cloud data or mesh data. Here, the measurement data is mesh data. Alternatively, the processing device 100 may convert the point cloud data into a triangular mesh model. The coordinate system setting unit 120 derives an initial reference coordinate system for the measurement data (S102). The initial reference coordinate system is a three-dimensional orthogonal coordinate system, and has an x-axis, a y-axis, and a z-axis that are orthogonal to each other. Details of this process will be described later.

[0043] The first determination unit 130 selects an axis (x, y, z) in the initial reference coordinate system (S103). That is, the first determination unit 130 selects one axis from the x-axis, y-axis, and z-axis. The first determination unit 130 determines whether the object 0 is rotationally symmetric about the selected axis (hereinafter also referred to as the selected axis) (S104). Rotational symmetry is a shape in which the shape when rotated around the selected axis matches the original shape.

[0044] If the object O is rotationally symmetric (YES in S104), the second judgment unit 140 judges whether or not the rotational symmetry has an angular period (S105). That is, the third judgment unit 150 judges whether or not the object O has a rotationally symmetric shape with an angular period. Rotational symmetry with an angular period is a shape in which the shape when rotated around a selected axis by a predetermined angle (an angle greater than 0° and less than 360°) matches the original shape. In other words, the shape when rotated around an axis by an angle other than the angle indicated by the angular period does not match the original shape.

[0045] If the rotational symmetry has an angular period (YES in S105), the modification unit 160 modifies the reference coordinate system (S106). The modification unit 160 changes the directions of two axes other than the selected axis in the initial reference coordinate system. This determines the directions of the two axes in a plane perpendicular to the selected axis. Then, the process proceeds to step S109. If the rotational symmetry does not have an angular period (NO in S105), the process proceeds to step S109.

[0046] If the object O is not rotationally symmetric (NO in S104), the third determination unit 150 determines whether the object O is plane symmetric (S107). Planar symmetry is a shape in which the shape of the object O when inverted on a plane including the selected axis matches the original shape. If the object O is plane symmetric (YES in S107), the correction unit 160 corrects the reference coordinate system (S108). The correction unit 160 changes the directions of two axes other than the selected axis in the initial reference coordinate system. Specifically, the correction unit 160 sets the direction of one axis other than the selected axis to be parallel to the symmetry plane (inverted plane). Then, the process proceeds to S109.

[0047] If the object O is not plane symmetric (NO in S107), the process proceeds to S109. Then, the first judgment unit 130 judges whether the process has been completed for all axes (S109). If the process has not been completed for all axes (NO in S109), the process returns to step S103, where the first judgment unit 130 selects an axis. That is, the processing device 100 performs the above process for the axes for which symmetry has not been determined. For example, the processes of S104 to S108 are performed in the order of the z-axis, y-axis, and x-axis. If the process has not been completed for all axes (YES in S109), the process ends.

[0048] The process of deriving the initial reference coordinate system in step S102 will be described with reference to Fig. 3. Fig. 3 is a flowchart for explaining the process in the coordinate system setting unit 120.

[0049] The coordinate system setting unit 120 projects the normals of all the faces of the mesh model onto a hemisphere (S201). The coordinate system setting unit 120 obtains normals (hereinafter also referred to as mesh normals) that are perpendicular to the faces of each mesh (also referred to as mesh faces). As described below, the coordinate system setting unit 120 derives initial reference coordinates based on the direction of the mesh normals. This makes it possible to easily determine an appropriate initial reference coordinate system. The number of mesh normals obtained here matches the number of mesh faces. In other words, if the number of mesh faces included in the mesh model of the object O is n (n is an integer equal to or greater than 2), the number of mesh normals is n.

[0050] The coordinate system setting unit 120 projects a normal on the hemisphere onto the hemisphere. For example, the coordinate system setting unit 120 generates a hemisphere that contains an object. The coordinate system setting unit 120 projects a mesh normal to a position perpendicular to the hemisphere. Specifically, the mesh normal is parallel to the normal of the hemisphere at a certain position (one point) on the hemisphere. The coordinate system setting unit 120 projects the normal to a position on the hemisphere where the mesh normal is parallel to the normal of the hemisphere. In this way, the coordinate system setting unit 120 projects the mesh normal onto the hemisphere according to the direction of the mesh normal. Two mesh normals that have the same direction are projected to the same position on the hemisphere.

[0051] The coordinate system setting unit 120 converts the normal distribution into latitude and longitude, and then discretizes and merges overlaps (S202). For example, the coordinate system setting unit 120 calculates the latitude and longitude of the projection position of the mesh normal on the hemisphere. The coordinate system setting unit 120 stores the latitude and longitude for each normal. Then, the coordinate system setting unit 120 merges normals with close latitude and longitude into one as overlapping normals. That is, when the latitude difference and longitude difference of two normals are equal to or less than a predetermined value, the two normals are merged into one. The latitude and longitude of the merged normal may be the average value of the latitude and longitude of the two normals. Alternatively, the latitude and longitude of the merged normal may be the latitude and longitude of one of the normals. For example, the number of normals before being merged can be n (n is an integer of 2 or more) and the number of normals after being merged can be m (m is an integer of 2 or more and equal to or less than n).

[0052] The coordinate system setting unit 120 creates a coordinate system for each normal line with the normal line as the z-axis (S203). Here, an xyz three-dimensional orthogonal coordinate system having an x-axis and a y-axis perpendicular to the z-axis is set. Coordinate systems are created as many times as the number m of normal lines after consolidation (m is an integer of 2 or more). In other words, the coordinate system setting unit 120 creates m coordinate systems. The direction of the z-axis differs depending on the projection position on the hemispherical surface. Therefore, the m coordinate systems have different axial directions.

[0053] The coordinate system setting unit 120 projects the vertices of the mesh model onto the xy plane to create a two-dimensional convex hull (S204). The two-dimensional convex hull is the smallest polygon that contains all given points. In other words, in the xy plane view, the two-dimensional convex hull contains the object O. The coordinate system setting unit 120 creates a two-dimensional convex hull that contains all the vertices of the mesh model on the xy plane of each coordinate system. Here, the coordinate system setting unit 120 can create the two-dimensional convex hull using a known algorithm. Here, since m coordinate systems are set, m two-dimensional convex hulls are created. The planes on which the m two-dimensional convex hulls are created are different planes.

[0054] The coordinate system setting unit 120 obtains a transformation that minimizes the area of ​​a bounding box that encloses the two-dimensional convex hull (S205). The bounding box is defined as a rectangle that includes the two-dimensional convex hull in the xy plane. The coordinate system setting unit 120 then generates a rectangle that includes the two-dimensional convex hull and has the smallest area as the bounding box. Here, a bounding box is obtained for each coordinate system. Thus, the coordinate system setting unit 120 generates m bounding boxes. The bounding box includes all vertices of the mesh model in the xy plane.

[0055] The coordinate system setting unit 120 obtains the range of the vertices of the mesh model in the z-axis direction, and obtains the volume of a three-dimensional bounding box (S206). As shown in FIG. 4, the three-dimensional bounding box is a rectangular parallelepiped that contains the object. In other words, the three-dimensional bounding box is a rectangular parallelepiped that contains all the vertices of the mesh model. The coordinate system setting unit 120 sets a three-dimensional bounding box whose base is a rectangle that encloses the two-dimensional convex hull in the xy plane. In other words, the rectangle corresponding to the bounding box obtained in step S205 defines the size of the three-dimensional bounding box in the x and y directions. The length from the vertex closest to the +z side to the vertex closest to the -z side defines the size of the three-dimensional bounding box in the z direction.

[0056] This allows the coordinate system setting unit 120 to obtain the volume of the three-dimensional bounding box. The coordinate system setting unit 120 calculates the product of the area of ​​the smallest rectangle enclosing the two-dimensional convex hull and the z-axis size as the volume of the three-dimensional bounding box. The coordinate system setting unit 120 determines a three-dimensional bounding box for each coordinate system and obtains its volume. As described above, since m coordinate systems are set, the coordinate system setting unit 120 calculates the volumes of m three-dimensional bounding boxes.

[0057] The coordinate system setting unit 120 sets the coordinate axes of the three-dimensional bounding box with the smallest volume as the axes of the reference coordinate system (S207). That is, the coordinate system setting unit 120 compares the volumes of the m three-dimensional bounding boxes to obtain the three-dimensional bounding box with the smallest volume. The coordinate system setting unit 120 selects the smallest three-dimensional bounding box from the m three-dimensional bounding boxes. Then, the directions of each side of the extracted three-dimensional bounding box are set as the axial directions. Therefore, the x-axis, y-axis, and z-axis are parallel to each side of the three-dimensional bounding box as shown in FIG. 4.

[0058] The coordinate system setting unit 120 sets the center point (origin) of the coordinate system to the center of the selected three-dimensional bounding box (S208). The coordinate system setting unit 120 sets a rectangular parallelepiped containing the object O as the three-dimensional bounding box. The coordinate system setting unit 120 then derives an initial reference coordinate system with the center of the rectangular parallelepiped as the origin and the directions of each side of the rectangular parallelepiped as x, y, and z axes. The coordinate system setting unit 120 replaces the axes in the order of the lengths of the sides of the three-dimensional bounding box (S209). For example, the x-axis, y-axis, and z-axis are set in order from the longest side. In this way, the coordinate system setting unit 120 can determine the initial reference coordinate system.

[0059] 5 is a flowchart for explaining an example of the judgment process in the first judgment unit 130 and the second judgment unit 140. Here, the axis selected in step S103 in the figure is assumed to be a. That is, the axis a is any one of the x-axis, y-axis, and z-axis in the initial reference coordinate system. In the following explanation, the axis a is assumed to be the z-axis, and the plane P perpendicular to the axis a is assumed to be the xy plane.

[0060] The first determination unit 130 creates an object model and cross-sectional shapes So, Sc of its convex hull on a plane P perpendicular to the axis a (S301). The cross-sectional shape So and the cross-sectional shape Sc are found by the intersection of the plane P and the object model. The cross-sectional shape So of the object is a shape that indicates the contour of the object on the plane P. The convex hull is the smallest polygon that contains the object (i.e., the cross-sectional shape So) on the plane P. The first determination unit 130 generates the cross-sectional shapes So, Sc by changing the position on the axis a. Therefore, for one object O, the first determination unit 130 generates a plurality of cross-sectional shapes So and a plurality of cross-sectional shapes Sc.

[0061] The first determination unit 130 obtains the distance r between the origin Po and the outermost intersection point of the straight line l and the cross-sectional shape Sc (or So) on the plane P (S302). The cross-sectional shape Sc and its r will be described with reference to FIG. 6. FIG. 6 shows the plane P. Here, the straight line l is a straight line that is parallel to the plane P as shown in FIG. 6 and passes through the origin Po of the plane P. The straight line l also starts from the origin P0. The angle between the straight line l and the x-axis on the plane P is θ. On the straight line l, the distance r indicates the distance from the origin Po to the outer shape of the cross-sectional shape Sc. When there are two or more intersection points between the straight line l and the cross-sectional shape Sc, the distance between the outermost intersection point and the origin Po is set to r.

[0062] As described above, the first determination unit 130 may use the cross-sectional shape So of the object model instead of the cross-sectional shape Sc. Therefore, in the following flow, it is also possible to replace the cross-sectional shape Sc with the cross-sectional shape So. When the cross-sectional shape So is used, the distance from the origin Po to the outline of the cross-sectional shape So on the straight line l is set as the distance r. In the following explanation, an example of processing the cross-sectional shape Sc will be explained. For this reason, as shown in FIG. 6, the distance from the origin Po to the intersection point between the straight line l and the cross-sectional shape Sc will be explained as the distance r.

[0063] Furthermore, the first determination unit 130 changes θ to obtain the distance r. That is, the first determination unit 130 calculates r over the entire circumference of a circle centered on the origin Po on the plane P. For example, the first determination unit 130 changes θ to δ θ interval to obtain r. This enables the first determination unit 130 to obtain a discrete function r(θ). For example, θ takes on discrete values ​​from 0° to 360°. r(θ) is a polar coordinate expression of the cross-sectional shape Sc. In other words, r(θ) corresponds to a function obtained by polar coordinate conversion of the cross-sectional shape Sc. Furthermore, the first determination unit 130 calculates r(θ) for each of a plurality of cross-sectional shapes Sc generated by changing the position on the axis a (here, the z position).

[0064] The first determination unit 130 normalizes r(θ) (S303). For example, the first determination unit 130 multiplies r(θ) by a constant coefficient so that the maximum value of r(θ) is 1. This makes r(θ) fall within the range of 0 to 1. The first determination unit 130 determines whether the value of r(θ) is constant (S304). That is, the first determination unit 130 determines whether the cross-sectional shape Sc is circular. More specifically, the first determination unit 130 determines whether the outer shape of the cross-sectional shape Sc is circular.

[0065] If r(θ) is constant (YES in S304), the first judgment unit 130 judges that the object is circular (rotationally symmetric) (S305). If it is judged to be circular, the processing of the first judgment unit 130 and the second judgment unit 140 ends. That is, in step S105 of FIG. 2, NO is judged. Examples of three-dimensional objects whose cross-sectional shape is circular include spheres, cylinders, cones, and rugby balls. In addition, there are bottles and the like whose diameter changes depending on the position of the axis a. These shapes are rotationally symmetric with respect to the central axis.

[0066] If r(θ) is not constant (NO in S304), the first determination unit 130 determines that the object is not circular (S306). Next, the second determination unit 140 determines whether the object has rotational symmetry with an angular period. That is, the determination process in step S105 in FIG. 2 is performed.

[0067] The second determination unit 140 performs a discrete Fourier transform on r'(θ) (S307). Here, r'(θ) is the average value r of r(θ) and r. ave Specifically, r'(θ)=r(θ)-r ave The average value r ave corresponds to the DC component of r. Moreover, the Fourier series (amplitude spectrum) R(ω) obtained by the discrete Fourier transform is shown in Fig. 7. The graph on the left side of Fig. 7 shows r'(θ), and the graph on the right side shows R(ω). When the cross-sectional shape Sc is a regular hexagon as shown in Fig. 6, peaks exist every 60° in R(w) shown in Fig. 7. Moreover, since the Fourier transform is performed on a real function in Fig. 7, the positive and negative values ​​are symmetrical.

[0068] The second determination unit 140 calculates the angle period ω at which the amplitude R is maximized from R(ω). max (S308). Here, ω max is between 0° and 180°. In FIG. 7, the maximum peak appears at R (60°), so ω max is 60° (=π / 3[rad]).

[0069] The second determination unit 140 calculates the difference (r(θ)−r(θ+ω) max The area E of the absolute value of r(θ) is calculated (S309). Here, the difference is the function r(θ) and the function r(θ) with an angular period ω max The function r(θ+ω max The absolute area E can be calculated using the following formula (1).

[0070]

number

[0071] The area E is found by integrating the absolute value with respect to θ in the range of 0° to 360°. In equation (1), since θ takes discrete values, the sum of the absolute values ​​of θ in the range of 0° to 360° is the area E. The second determination unit 140 determines whether the area E is less than a threshold value (S310). That is, the third determination unit 150 compares the area E with a preset threshold value.

[0072] If it is less than the threshold value (YES in S310), the second determination unit 140 determines whether the angular period ω max In other words, it is determined that the shape has a rotational symmetry (line symmetry) with an angular period ω max Since the difference in the functions when shifted by only one is small, the second determination unit 140 determines that the shape is rotationally symmetric (line symmetric) with an angular period. If the difference is not less than the threshold value (NO in S310), the second determination unit 140 determines that the shape is not rotationally symmetric (line symmetric) with an angular period (S312).

[0073] Figure 8 shows the relationship between r(θ) and r(θ+ω max) is shown. In Fig. 8, the graph on the left shows a shape that is not rotationally symmetric, and the graph on the right shows a shape that is rotationally symmetric. In the case of a shape that is not rotationally symmetric, r(θ) and r(θ+ω max ) is not 0, the area E calculated by formula (1) is large. For a rotationally symmetric shape with an angular period, the difference between r(θ) and r(θ+ω max ) are almost the same function. Therefore, for any θ, r(θ) and r(θ+ω max ) is approximately 0, and the area E is approximately 0. Therefore, the second determination unit 140 can determine whether or not there is rotational symmetry with an angular period by comparing the area E with a preset threshold value.

[0074] Note that multiple cross-sectional shapes Sc are generated by changing the position on the axis a. The determination of rotational symmetry is performed for each cross-sectional shape Sc. Therefore, if the value of r(θ) is constant for all cross-sectional shapes Sc, they are determined to be rotationally symmetric without angular period. If the area E of one or more cross-sectional shapes Sc is not less than the threshold value, they are determined to not be rotationally symmetric. If the area E of some cross-sectional shapes Sc is less than the threshold value and r(θ) is constant for all remaining cross-sectional shapes, they are determined to be rotationally symmetric with angular period. In addition, for all cross-sectional shapes Sc, even if the area E is less than the threshold value, the angular period ω max In the case where the cross-sectional shapes Sc are different, the cross-sectional shape may not be rotationally symmetric with the angular period. max If the two are misaligned and do not overlap, the rotational symmetry with the angular period is not achieved. For example, in a certain cross-sectional shape, the angular period ω max is 60°, and in the remaining cross-sectional shapes, the angular period ω max is 72°, the cross-sectional shape does not have rotational symmetry with an angular period. max is 60°, and in the remaining cross-sectional shapes, the angular period ω max When the angle is 30°, the rotational symmetry has an angular period, i.e., 6-fold rotational symmetry.

[0075] If the object has rotational symmetry with an angular period (S311), the correction unit 160 corrects the initial reference coordinate system as in step S106 of FIG. 2. The correction unit 160 may correct the axis direction of the initial reference coordinate system based on the peak surface. For example, the angle θ at which r(θ) is maximized max The correction unit 160 sets the reference coordinate system so that the direction of θ max where ω max is greater than or equal to 0° and less than 360°.

[0076] r(θ max The position of the cross-sectional shape Sc or the cross-sectional shape So at which the angle θ max The line inclined by θ is the axis b. max is 0°. Axis b is a straight line passing through the origin and the peak position. The plane including the selected axis (axis a) and axis b is the peak plane. If there are multiple angles θ at which r(θ) is maximized, axis b can be set based on one of the peak positions.

[0077] After step S312, the third determination unit 150 determines whether or not there is plane symmetry, as in step S107. An example of determining whether or not there is plane symmetry will be described with reference to Figs. 9 to 11. Figs. 9 and 10 are flow charts showing an example of the determination process. Fig. 11 is a diagram showing a plane-symmetric cross-sectional shape. In the following description, the process is performed for the cross-sectional shape Sc of the convex hull, but it is also possible to perform the process for the cross-sectional shape So of the object. Therefore, in the following flow, it is also possible to replace the cross-sectional shape Sc with the cross-sectional shape So.

[0078] The third determination unit 150 determines the origin P of the plane P. O A straight line L(φ) passing through the φ(S401). Thus, a number of straight lines L(φ) are set on the plane P. Here, as shown in FIG. 11, the straight lines L(φ) are straight lines that pass through the origin Po on the plane P and make an angle φ with the x-axis. In other words, the angle φ is the angle between the x-axis and the straight lines L(φ) on the plane P. Moreover, φ is in the range of 0° or more and less than 180°. The straight lines L(φ) are set so as to cross the cross-sectional shape Sc.

[0079] The third determination unit 150 takes sample points Qm at regular intervals δm on the straight line L(φ) (S402). This allows multiple points on the straight line (φ) to be sampled. The third determination unit 150 records the xy coordinates of each sample point Qm. FIG. 11 shows the straight line L(φ) and some of the sample points Qm on the straight line L(φ). Only three sample points placed on one straight line L(φ) are shown here. The third determination unit 150 also draws multiple straight lines L(φ) by changing the angle φ. The third determination unit 150 finds multiple sample points Qm on each straight line L(φ). In the following process, unless otherwise specified, the same process is performed for each cross-sectional shape.

[0080] The third determination unit 150 draws an orthogonal straight line K(Qm) that passes through Qm and is perpendicular to the straight line L(φ) for all sample points Qm (S403). The third determination unit 150 determines intersections Qmu and Qml between K(Qm) and the cross-sectional shape Sc (S404). As shown in FIG. 11, K(Qm) is a straight line that crosses the cross-sectional shape Sc of the convex hull, so K(Qm) intersects with the cross-sectional shape Sc at two points. Therefore, as shown in FIG. 12, the intersection point on the positive side between K(Qm) and the cross-sectional shape Sc is Qmu, and the intersection point on the negative side is Qml. In addition, when K(Qm) intersects with the cross-sectional shape Sc at three or more points, the outermost intersection point on the positive side is Qmu, and the outermost intersection point on the negative side is Qml.

[0081] The third determination unit 150 determines the distances du and dl from the sample point Qm to the intersections Qmu and Qml (S405). As shown in FIG. 12, the third determination unit 150 determines the distance between the sample point Qm and the intersection Qmu as du, and the distance between the sample point Qm and the intersection Qml as dl.

[0082] Next, the third determination unit 150 calculates the signed distance difference d=du-dl (S406). ave The third determination unit 150 calculates the average value d for multiple sample points Qm on one line L(φ) by calculating the sum of d and dividing the sum by the number of sample points. ave The third judgment unit 150 calculates d and d ave The sum of the absolute values ​​of the differences d sum The sum d is calculated (S408). sum can be calculated using the following formula (2).

[0083]

number

[0084] The third determination unit 150 calculates the sum d sum The angle φ at which min (S409). As described above, the signed distance difference d and the average value d ave , the sum d sum are obtained for each angle φ. In other words, the third determination unit 150 performs the above process for each straight line L(φ) to obtain the signed distance difference d, the average value d ave , the sum d sum The third determination unit calculates the sum d sum Compare and find the minimum sum d sum Then, find the angle φ of the line L(φ). The angle φ at which the cross-sectional shape Sc is closest to line symmetry (inversion symmetry) with respect to the line L(φ) is the angle φ min is set as:

[0085] The third determination unit 150 determines the angle φ min The average value of the signed distance difference d when ave_min and the sum d sum_min The sum d is calculated (S410). sum_min The third determination unit 150 determines whether the sum d sum_min is compared with a preset threshold value.

[0086] Sum d sum_min is not less than the threshold value (NO in S411), the third determination unit 150 determines that the shape is not symmetrical (S412). That is, the cross-sectional shape Sc (or the cross-sectional shape So) is neither rotationally symmetrical nor plane symmetrical about the axis a. Therefore, the object has no symmetry about the axis a.

[0087] Sum d sum_min If is less than the threshold value (YES in S411), the correction unit 160 min and d ave_min Based on this, the axis b is set (S413). Specifically, the axis b has an inclination of an angle φ with respect to the x-axis and is d ave_min Figure 12 shows axis b on plane P perpendicular to the z-axis. Axis b is a straight line L (φmin) separated by d ave_min This corresponds to a straight line translated by a distance x. The position of the origin Po is also translated in the same way. The remaining axis c is a direction perpendicular to the axis b and perpendicular to the axis a. The third determination unit 150 determines that the object O is plane-symmetric with respect to a plane (inversion plane) including the axes a and b (S414). In this way, when the object O has plane symmetry, the modification unit 160 modifies the axial direction of the initial reference coordinate system so that the object O has an axis along the inversion plane.

[0088] The process of determining plane symmetry shown in Fig. 9 and Fig. 10 is performed for each cross-sectional shape Sc. Then, for all cross-sectional shapes Sc, the sum d sum_min is less than the threshold value and the axis b is common, it is determined that they are plane symmetric. Note that the processes shown in Figs. 3 to 12 are examples of the determination process, and the present embodiment is not limited to the above processes.

[0089] System 1 performs the above judgment process for each of the three axes of the initial reference coordinate system. In other words, symmetry is judged for each of the three axes. If it is judged to be a rotationally symmetric shape without angular period on two or more axes, it is judged to be a spherical shape. If it is rotationally symmetric about one axis and plane symmetric about another axis, it becomes an object with two types of symmetry, such as a cylinder or a rugby ball shape. If it is rotationally symmetric about one axis, it can be determined whether it is symmetry with angular period (discrete symmetry) or symmetry with angular period. Examples of discretely symmetric cross-sectional shapes include regular polygons and star shapes. Various symmetries can be judged using the above algorithm.

[0090] In summary, the processing device 100 executes the following processes (1) to (6). (1) Derive an initial reference coordinate system based on measurement data that indicates the three-dimensional shape of the object. (2) selecting an axis of the initial reference coordinate system; (3) determining whether or not there is rotational symmetry about the axis; (4) determining whether or not the rotational symmetry has an angular period with respect to the axis; (5) if there is no rotational symmetry, determining plane symmetry about the axis; (6) By changing the axis selected in (2), the processes in (3) to (5) are performed for each of the three axes of the initial reference coordinate system.

[0091] In this way, an appropriate coordinate system can be set. Then, a 3D model of an object is expressed using the 3D coordinate system set as above. In other words, a 3D model showing the 3D shape of an object is expressed in an appropriate coordinate system. For example, the position coordinates of a point cloud or mesh showing a 3D model are shown in an appropriate coordinate system. Various symmetry information can be annotated. A 3D shape model can be created with coordinate axes that take symmetry into account.

[0092] The processing device 100 derives initial reference coordinates from the three-dimensional model. The processing device 100 can efficiently perform calculations by sequentially searching for rotational symmetry and plane symmetry on each axis. Information on the coordinate system setting and symmetry for the three-dimensional shape model of the object can be annotated. A three-dimensional model in an appropriate coordinate system can be efficiently generated.

[0093] In the case of an object having symmetry, the axes of the coordinate system define the symmetry axis or symmetry plane (inversion plane). In the case of rotational symmetry having the above-mentioned angular period or plane symmetry, the correction unit 160 corrects the initial reference coordinate system. By using the axes of the corrected coordinate system, the position and orientation of the object can be appropriately changed. The learning data can be created simply and appropriately. Furthermore, the learning data can be appropriately and simply annotated with position and orientation information indicating the position and orientation. It is possible to set a coordinate system useful for estimating the six-axis position and orientation in the estimator.

[0094] The coordinate system setting unit 120 may derive the initial reference coordinates based on the direction of the normal to each mesh of the mesh model representing the three-dimensional shape. This makes it possible to easily determine an appropriate initial reference coordinate system.

[0095] The coordinate system setting unit 120 sets a rectangular parallelepiped containing the object O as a three-dimensional bounding box. The coordinate system setting unit 120 then derives an initial reference coordinate system with the center of the rectangular parallelepiped as the origin and the directions of each side of the rectangular parallelepiped as the x, y, and z axes.

[0096] When the object has plane symmetry, the correction unit 160 corrects the axial directions of the initial reference coordinate system so as to have axes along the inversion plane.

[0097] When the object has rotational symmetry with an angular period, the correction unit 160 corrects the axial directions of the initial reference coordinate system based on the peak surface.

[0098] The above processing method can be implemented by a computer program or hardware. In other words, the processing device 100 executes a predetermined program to function as a generating device or a generating system. FIG. 13 shows an example of a hardware configuration of the processing device 100. The processing device 100 includes a processor 10, a memory 20, and an interface 30. The memory 20 stores programs, various parameters, measurement data, and the like. The processor 10 executes the programs stored in the memory 20. The interface 30 transmits data to the sensor 200. The interface 30 also receives data from the sensor 200.

[0099] The method according to the present embodiment can be executed by the processor 10 of the processing device 100 executing a program. The processing device 100 only needs to have at least one processor 10. The one or more processors 10 execute a program stored in a memory to perform the above-mentioned processing. The processing device 100 is not limited to being a single physical device, and may be distributed among multiple devices. In other words, the above-mentioned method may be executed by multiple devices performing distributed processing.

[0100] A part or all of the above processes may be executed by a computer program. That is, the control of the processing device 100 is executed by a control computer that configures the processing device 100 executing the program. The above-mentioned program includes a group of instructions (or software code) that causes the computer to perform one or more functions described in the embodiment when the program is loaded into the computer. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or the tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or the communication medium includes electrical, optical, acoustic, or other forms of propagating signals.

[0101] The invention made by the inventor has been specifically described above based on an embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the gist of the invention. [Explanation of symbols]

[0102] 1 System 10 Processors 20 Memory 30 Interface 100 Processing equipment 110 Measurement data acquisition unit 120 Coordinate system setting section 130 1st Judgment Department 140 Second Judgment Section 150 Third Judgment Section 160 Correction Department 200 Sensors 210 Table 220 Depth Sensor

Claims

1. At least one processor (1) deriving an initial reference coordinate system based on measurement data indicating a three-dimensional shape of an object; (2) selecting an axis of the initial reference coordinate system; (3) determining whether or not there is rotational symmetry with respect to the axis; (4) determining whether or not the rotational symmetry has an angular period with respect to the axis; (5) if there is no rotational symmetry, determining plane symmetry about the axis; (6) A method for generating a three-dimensional model, comprising: changing the axis selected in (2) above; and performing the processes (3) to (5) above for each of the three axes of the initial reference coordinate system.

2. The method for generating a three-dimensional model according to claim 1 , further comprising the step of correcting the initial reference coordinate system when there is rotational symmetry having the angular period or when there is planar symmetry.

3. 3. The method for generating a three-dimensional model according to claim 1, further comprising the step of: deriving the initial reference coordinate system based on a direction of a normal to each mesh of a mesh model representing the three-dimensional shape.

4. 3. A method for generating a three-dimensional model according to claim 1 or 2, further comprising the steps of: defining a rectangular parallelepiped containing the object as a three-dimensional bounding box; deriving an initial reference coordinate system with the center of the rectangular parallelepiped as the origin and each side direction of the rectangular parallelepiped as the x, y and z axes;

5. 3. The method for generating a three-dimensional model according to claim 1, wherein, when the model has planar symmetry, axial directions of the initial reference coordinate system are corrected so as to have axes along the inverted plane.

6. The method for generating a three-dimensional model according to claim 1 or 2, wherein, when the model has rotational symmetry with the angular period, the axial directions of the initial reference coordinate system are corrected based on a peak surface.

7. A system for generating a three-dimensional model comprising at least one processor, The processor, (1) deriving an initial reference coordinate system based on measurement data indicating a three-dimensional shape of an object; (2) selecting an axis of the initial reference coordinate system; (3) determining whether or not there is rotational symmetry with respect to the axis; (4) determining whether or not there is rotational symmetry with an angular period about the axis; (5) if there is no rotational symmetry, determining plane symmetry about the axis; (6) A three-dimensional model generating system, which performs the processes (3) to (5) for each of the three axes of the initial reference coordinate system by changing the axis selected in (2).

8. The three-dimensional model generating system according to claim 7 , wherein the initial reference coordinate system is modified when there is rotational symmetry having the angular period or when there is planar symmetry.

9. 9. The three-dimensional model generating system according to claim 7, wherein the initial reference coordinate system is derived based on a direction of a normal to each mesh of a mesh model representing the three-dimensional shape.

10. 9. A three-dimensional model generation system as described in claim 7 or 8, which derives an initial reference coordinate system in which a rectangular parallelepiped containing the object is defined as a three-dimensional bounding box, the center of the rectangular parallelepiped is defined as the origin, and each side direction of the rectangular parallelepiped is defined as an x, y, and z axis.

11. 9. The system for generating a three-dimensional model according to claim 7, wherein, when the system has planar symmetry, the axial directions of the initial reference coordinate system are corrected so as to have axes along the inverted plane.

12. 9. The system for generating a three-dimensional model according to claim 7, wherein, when the three-dimensional model has rotational symmetry with the angular period, the axial directions of the initial reference coordinate system are corrected based on a peak surface.

13. A program for causing a computer to execute a three-dimensional model generation method, The method for producing the same comprises: (1) deriving an initial reference coordinate system based on measurement data indicating a three-dimensional shape of an object; (2) selecting axes of the initial reference coordinate system; (3) determining whether or not there is rotational symmetry with respect to the axis; (4) determining whether or not the rotational symmetry has an angular period with respect to the axis; (5) if there is no rotational symmetry, determining plane symmetry about the axis; (6) A program comprising a step of performing the processes (3) to (5) for each of the three axes of the initial reference coordinate system by changing the axis selected in (2).

Images