Information Processing Method, Information Processing System, Program

Abstract

To enable fitting substantially without directly using a reference object model, which is a three-dimensional point cloud data model, for fitting, and to make it possible to compare an actual object model with a reference object model. [Solution] The information processing method, which is the main invention of the present invention, includes a step of acquiring reference 3D model data from a reference object by a sensor, a step of acquiring any number of reference point coordinate information of three or more from the reference 3D model data by a reference point coordinate acquisition unit, a step of acquiring corresponding 3D model data from a measurement object by a sensor, a step of setting a reference search range based on the reference point coordinate information by a reference search range setting unit, and a step of acquiring corresponding point coordinate information within the reference search range in the corresponding 3D model data by a corresponding point coordinate information acquisition unit. and acquiring the information.

Description

Technical Field

[0001] The present invention relates to an information processing method, an information processing system, and a program.

Background Art

[0002] Conventionally, a target object (e.g., a workpiece) is scanned in advance to create a reference target object model, and the actual target object is scanned to create an actual target object model, and position deviation correction is performed according to the comparison result with the reference target object model. A technique for this is known. For example, Patent Document 1 discloses a technique for obtaining a position deviation amount by comparing a reference image and an input image with each other.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004]

[0005]

[0005] That is, for example, when performing shape comparison or the like using both models, the shape of the actual object model is within a range of shape change that allows fitting to the reference object model, or further interpolation may be required for fitting.

[0006] The present invention has been made in view of such a background, and it is possible to substantially perform fitting without directly using a reference object model that is a three-dimensional point cloud data model and aims to provide a technique capable of comparing an actual object model and a reference object model.

Means for Solving the Problems

[0007] The main invention of the present invention for solving the above problems is a step of obtaining reference three-dimensional model data from a reference object by a sensor, a step of obtaining reference point coordinate information of three or more arbitrary numbers from the reference three-dimensional model data by a reference point coordinate acquisition unit, a step of obtaining corresponding three-dimensional model data from a measurement object by a sensor, a step of setting a reference search range based on the reference point coordinate information by a reference search range setting unit, and a step of obtaining corresponding point coordinate information within the reference search range in the corresponding three-dimensional model data by a corresponding point coordinate information acquisition unit, and is an information processing method characterized by including the above steps.

[0008] Regarding other problems disclosed in the present application and their solutions, they will be more clearly described in the column of the embodiments of the invention and the drawings.

Effects of the Invention

[0009] According to the present invention, it is possible to substantially perform fitting without directly using a reference object model that is a three-dimensional point cloud data model It becomes possible to substantially fit without using it for the group, and the actual object model and the reference object model are made comparable with each other.

Brief Description of Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0011] The content of the embodiments of the present invention will be listed and described. The present invention has, for example, the following configuration and includes.

[0012] [Item 1] An information processing method, comprising: a step of acquiring reference three-dimensional model data from a reference object by a sensor; a step of acquiring coordinate information of three or more arbitrary reference points from the reference three-dimensional model data by a reference point coordinate acquisition unit; a step of acquiring corresponding three-dimensional model data from a measurement object by a sensor; a step of setting a reference search range based on the reference point coordinate information by a reference search range setting unit; a step of acquiring corresponding point coordinate information within the reference search range in the corresponding three-dimensional model data by a corresponding point coordinate information acquisition unit; and including, characterized in that it is an information processing method. [Item 2] The information processing method according to Item 1, wherein each of the three-dimensional model data is three-dimensional point cloud data. characterized in that it is an information processing method. [Item 3] The information processing method according to any one of Items 1 or 2, wherein the step of acquiring the reference point coordinate information includes: a step of detecting a part having a specific shape in the reference three-dimensional model data by a specific shape part detection unit and acquiring coordinate information of the part as reference point coordinate information. characterized in that it is an information processing method. [Item 4] The information processing method according to any one of Items 1 to 3, ​​​The step of calculating a reference plane including the reference point coordinates in the reference three-dimensional model data by a reference point coordinate plane calculation unit and the step of obtaining reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane, which is set as a reference value in the predetermined coordinate axis direction on the reference plane by a reference difference value information acquisition unit and further includes: The step of obtaining reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane, which is set as a reference value in the predetermined coordinate axis direction on the reference plane by a reference difference value information acquisition unit and further includes: An information processing method characterized by the above. [Item 5] The information processing method according to any one of Items 1 to 3, including the step of calculating a corresponding plane including the corresponding point coordinates in the corresponding three-dimensional model data by a corresponding point coordinate plane calculation unit and the step of obtaining corresponding difference value information between the corresponding three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane, which is set as a reference value in the predetermined coordinate axis direction on the corresponding plane by a corresponding difference value information acquisition unit and further includes: The step of obtaining corresponding difference value information between the corresponding three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane, which is set as a reference value in the predetermined coordinate axis direction on the corresponding plane by a corresponding difference value information acquisition unit and further includes: An information processing method characterized by the above. [Item 6] The information processing method according to any one of Items 1 to 3, including the step of calculating a reference plane including the reference point coordinates in the reference three-dimensional model data by a reference point coordinate plane calculation unit and the step of obtaining reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane, which is set as a reference value in the predetermined coordinate axis direction on the reference plane by a reference difference value information acquisition unit and further includes: The step of obtaining reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane, which is set as a reference value in the predetermined coordinate axis direction on the reference plane by a reference difference value information acquisition unit and further includes: including the step of calculating a corresponding plane including the corresponding point coordinates in the corresponding three-dimensional model data by a corresponding point coordinate plane calculation unit and the step of obtaining corresponding difference value information between the corresponding three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane, which is set as a reference value in the predetermined coordinate axis direction on the corresponding plane by a corresponding difference value information acquisition unit and further includes: and obtaining corresponding difference value information between the corresponding three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane and further includes the step of obtaining corresponding difference value information between the corresponding three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane An information processing method characterized by the above. [Item 7] The information processing method according to Item 6, wherein a difference value information comparison unit further includes the step of comparing the reference difference value information and the corresponding difference value information with each other and calculating comparison result information of the ratio An information processing method characterized by the above. [Item 8] The information processing method according to Item 7, wherein the comparison result information is the deviation amount information of both obtained by further calculating the difference between the value indicated by the reference difference value information and the value indicated by the corresponding difference value information. Further, for example, based on the allowable range information of the deviation amount set by the user a flag indicating whether it is within or outside the allowable range may be set . An information processing method characterized by the above. [Item 9] The information processing method according to Item 8, wherein the difference value information comparison unit further includes the step of comparing the allowable range information of the deviation amount set by the user with the deviation amount information and displaying at least a part of the corresponding three-dimensional model data in different colors according to whether it is within or outside the allowable range An information processing method characterized by the above. [Item 10] An information processing system, comprising a three-dimensional model data acquisition unit that acquires reference three-dimensional model data and corresponding three-dimensional model data from a reference object and a measurement object by a sensor a reference point coordinate acquisition unit that acquires reference point coordinate information of three or more arbitrary numbers from the reference three-dimensional model data and a reference search range setting unit that sets a reference search range based on the reference point coordinate information ​​​​​ A corresponding point coordinate information acquisition unit that acquires corresponding point coordinate information within a reference search range in the corresponding three-dimensional model data; and, an information processing system characterized by including the above. [Item 11] A program for causing a computer to execute an information processing method, wherein the program, as the information processing method, includes steps of: acquiring reference three-dimensional model data from a reference object by a sensor; acquiring coordinate information of three or more arbitrary reference points from the reference three-dimensional model data by a reference point coordinate acquisition unit; acquiring corresponding three-dimensional model data from a measurement object by a sensor; setting a reference search range based on the reference point coordinate information by a reference search range setting unit; acquiring corresponding point coordinate information within the reference search range in the corresponding three-dimensional model data by a corresponding point coordinate information acquisition unit; and causing a computer to execute the above. A program characterized by the above.

[0013] <Details of Embodiment 1> A specific example of an information processing system 100 according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, and is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included. In the following description, in the accompanying drawings, the same or similar elements are given the same or similar reference numerals and names, and duplicate descriptions regarding the same or similar elements in the description of each embodiment may be omitted. Also, the features shown in each embodiment are applicable to other embodiments as long as they do not conflict with each other.

[0014] FIG. 1 is a diagram showing an example of the information processing system 100 of the present embodiment. As shown in FIG. 1 the information processing system 100 of the present embodiment includes a terminal 1, a work robot 2, and a con troller 3. The work robot 2 has at least an arm 21, a tool 22, and a sensor 23. The terminal 1, the controller 3, and the work robot 2 are connected to each other so as to be communicable by wire or wireless.

[0015] <Terminal 1> FIG. 2 is a diagram showing the hardware configuration of the terminal 1. The terminal 1 may be a general-purpose computer such as a personal computer, or may be logically realized by cloud computing . Note that the illustrated configuration is an example, and it may have other configurations. For example, some functions provided in the processor 10 of the terminal 1 may be executed by an external server or another terminal. The terminal 1 includes at least a processor 10, a memory 11, a storage 12, a transmission / reception unit 13 , an input / output unit 14, etc., and these are electrically connected to each other through a bus 15.

[0016] The processor 10 controls the operation of the entire terminal 1, and performs at least information processing necessary for controlling transmission / reception of data, etc. with the work robot 2, and executing and authenticating an application. For example, the processor 10 is a computing device such as a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes programs for the present system stored in the storage 12 and expanded in the memory 11 to perform each information processing.

[0017] ng Unit) and / or a GPU (Graphics Processing U nit), and executes programs for the present system stored in the storage 12 and expanded in the memory 11 to perform each information processing. ​

[0018] Memory 11 includes a main memory composed of a volatile memory device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory composed of a non-volatile memory device such as a flash memory or an HDD (Hard Disc Drive). Memory 11 is used as a work area of the processor 10, and stores a BIOS (Basic Input / Output System) executed when the terminal 1 is started, and various setting information and the like. y) and an auxiliary memory composed of a non-volatile memory device such as a flash memory or an HDD (Hard Disc Drive). Memory 11 is used as a work area of the processor 10, and stores a BIOS (Basic Input / Output System) executed when the terminal 1 is started, and various setting information and the like. 11 is used as a work area of the processor 10, and also stores a BIOS (Basic Input / Output System) executed when the terminal 1 is started, and various setting information and the like. and stores various setting information and the like.

[0019] Storage 12 stores various programs such as application programs. A database storing data used for each process may be constructed in Storage 12. It may also be possible to construct a database storing data used for each process in Storage 12.

[0020] The transceiver unit 13 connects the terminal 1 to at least the work robot 2 and performs transmission and reception of data and the like according to the instructions of the processor. Note that the transceiver unit 13 may be configured by wire or wirelessly, and when it is wireless, for example, it may be configured by a short-range communication interface such as WiFi, Bluetooth (registered trademark), and BLE (Bluetooth Low Energy). The transceiver unit 13 connects the terminal 1 to at least the work robot 2 and performs transmission and reception of data and the like according to the instructions of the processor. Note that the transceiver unit 13 may be configured by wire or wirelessly, and when it is wireless, for example, it may be configured by a short-range communication interface such as WiFi, Bluetooth (registered trademark), and BLE (Bluetooth Low Energy). LE (Bluetooth Low Energy). LE (Bluetooth Low Energy).

[0021] The input / output unit 14 is composed of, for example, an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse) when the terminal 1 is composed of a personal computer, and is composed of an information input / output device such as a touch panel when the terminal 1 is composed of a smartphone or a tablet terminal. The input / output unit 14 is composed of, for example, an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse) when the terminal 1 is composed of a personal computer, and is composed of an information input / output device such as a touch panel when the terminal 1 is composed of a smartphone or a tablet terminal. The input / output unit 14 is composed of, for example, an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse) when the terminal 1 is composed of a personal computer, and is composed of an information input / output device such as a touch panel when the terminal 1 is composed of a smartphone or a tablet terminal.

[0022] ​​​​Bus 15 is commonly connected to each of the above elements and transmits, for example, address signals, data signals, and various types of control signals.

[0023] <Working robot 2> Returning to FIG. 1, the working robot 2 according to this embodiment will be described.

[0024] As described above, the working robot 2 includes an arm 21, a tool 22, and a sensor 23. Note that the illustrated configuration is an example and is not limited to this configuration.

[0025] The arm 21 is controlled in its operation by the terminal 1 based on a three-dimensional robot coordinate system. Also, the arm 21 may further include a controller 3 connected to the working robot 2 by wire or wirelessly, and its operation may be controlled thereby.

[0026] The tool 22 is controlled in its operation by the terminal 1 based on a three-dimensional tool coordinate system. Also, the configuration of the tool 22 may include any tool according to the application, for example, a welding torch, a paint spraying device for painting, a gripping device, an excavating device, a polishing device, etc.

[0027] The sensor 23 performs sensing of an object based on a three-dimensional sensor coordinate system. The sensor 2 3 is, for example, a laser sensor that operates as a three-dimensional scanner, and acquires three-dimensional model data 40 of the object by sensing. The three-dimensional model data 40 is, for example, three-dimensional point cloud data as shown in FIGS. 4 and 5, and each point data has coordinate information in the sensor coordinate system, and it is possible to grasp the shape of the object from the point cloud. Note that the sensor 23 is not limited to a laser sensor, and may be, for example, an image sensor using a stereo method or the like. It may be a sensor independent of the work robot, as long as coordinate information can be obtained in a three-dimensional sensor coordinate system. Also, for the sake of concretizing the explanation, hereinafter a configuration using three-dimensional point cloud data as the three-dimensional model data 40 will be described as an example.

[0028] Note that before the work, a predetermined calibration is performed to associate the robot coordinate system, the tool coordinate system, and the sensor coordinate system with each other. For example, when the user designates a position (coordinate) based on the sensor coordinate system, the operation of the arm 21 and the tool 22 may be controlled based on the corresponding positions.

[0029] <Function of Terminal 1> FIG. 3 is a block diagram illustrating the functions implemented in the terminal 1. In the present embodiment, the processor 10 of the terminal 1 has a three-dimensional model data acquisition unit (three-dimensional point cloud data acquisition unit) 101, a three-dimensional model data display unit (three-dimensional point cloud data display unit) 102, a reference point coordinate acquisition unit 103, a specific shape part detection unit 104, a reference search range setting unit 105, and a corresponding point coordinate information acquisition unit 106. Also, the storage 12 of the terminal 1 has a three-dimensional model data storage unit ( three-dimensional point cloud data storage unit) 121, a reference point coordinate information storage unit 122, and a corresponding point coordinate information storage

[0030] unit 123. The three-dimensional point cloud data acquisition unit 101 controls the work robot 2 according to an instruction from the input / output unit 14 of the terminal 1, and acquires the three-dimensional point cloud data of the object by the sensor 23. The acquired three-dimensional point cloud data is, for example, three-dimensional coordinate information data based on the sensor coordinate system, and is stored in the three-dimensional point cloud data storage unit 121.

[0031] ​​​​The three-dimensional point group data display unit 102 displays the three-dimensional point group data acquired by the three-dimensional point group data acquisition unit 101 on the input / output unit 14 of the terminal 1 as exemplified in FIG. 4, for example. The user can view the displayed three-dimensional point group data from an arbitrary direction. In this case, as described above, the input / output unit 14 is composed of, for example, an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse) when the terminal 1 is composed of a personal computer, and is composed of an information input / output device such as a touch panel when the terminal 1 is composed of a smartphone or a tablet terminal. The input / output unit 14 can specify the above arbitrary direction. The user can view the displayed three-dimensional point group data from an arbitrary direction. In this case, as described above, the input / output unit 14 is composed of, for example, an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse) when the terminal 1 is composed of a personal computer, and is composed of an information input / output device such as a touch panel when the terminal 1 is composed of a smartphone or a tablet terminal. The input / output unit 14 can specify the above arbitrary direction. When the terminal 1 is composed of a personal computer, it is composed of an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse). When the terminal 1 is composed of a smartphone or a tablet terminal, it is composed of an information input / output device such as a touch panel. When the terminal 1 is composed of a personal computer, it is composed of an information output device (e.g., a display) and an information input device (e.g., a keyboard and a mouse). When the terminal 1 is composed of a smartphone or a tablet terminal, it is composed of an information input / output device such as a touch panel. The input / output unit 14 can specify the above arbitrary direction. The input / output unit 14 can specify the above arbitrary direction.

[0032] The reference point coordinate acquisition unit 103 acquires arbitrary numbers of reference point coordinate information, for example, according to a predetermined operation of the user, based on reference three-dimensional model data 41 (e.g., reference three-dimensional point group data) as exemplified in FIG. 7 and the like. As a specific example, for example, reference origin coordinate information with the origin as the origin, X reference point coordinate information with a reference point in the X direction from the origin, and Y reference point coordinate information with a reference point in the Y direction from the origin may be acquired. The acquisition method of each reference coordinate information may be, for example, that the user directly selects a predetermined point from the reference three-dimensional point group data 41 displayed by the three-dimensional point group data display unit 102 on the input / output unit 14 of the terminal 1, or a predetermined point at a specific-shaped part on the reference object may be determined by a predetermined calculation. The specific-shaped part may be, for example, a hole or a protrusion such as a conical shape or a cylindrical shape. For the predetermined calculation method, for example, a hole (hole part) may be detected by the method described later, and the center coordinate information of the hole may be used as the reference coordinate information, or the tip of the protrusion may be determined by a known method. The reference point coordinate acquisition unit 103 acquires arbitrary numbers of reference point coordinate information, for example, according to a predetermined operation of the user, based on reference three-dimensional model data 41 (e.g., reference three-dimensional point group data) as exemplified in FIG. 7 and the like. As a specific example, for example, reference origin coordinate information with the origin as the origin, X reference point coordinate information with a reference point in the X direction from the origin, and Y reference point coordinate information with a reference point in the Y direction from the origin may be acquired. The acquisition method of each reference coordinate information may be, for example, that the user directly selects a predetermined point from the reference three-dimensional point group data 41 displayed by the three-dimensional point group data display unit 102 on the input / output unit 14 of the terminal 1, or a predetermined point at a specific-shaped part on the reference object may be determined by a predetermined calculation. The acquisition method of each reference coordinate information may be, for example, that the user directly selects a predetermined point from the reference three-dimensional point group data 41 displayed by the three-dimensional point group data display unit 102 on the input / output unit 14 of the terminal 1, or a predetermined point at a specific-shaped part on the reference object may be determined by a predetermined calculation. The specific-shaped part may be, for example, a hole or a protrusion such as a conical shape or a cylindrical shape. The specific-shaped part may be, for example, a hole or a protrusion such as a conical shape or a cylindrical shape. For the predetermined calculation method, for example, a hole (hole part) may be detected by the method described later, and the center coordinate information of the hole may be used as the reference coordinate information, or the tip of the protrusion may be determined by a known method. For the predetermined calculation method, for example, a hole (hole part) may be detected by the method described later, and the center coordinate information of the hole may be used as the reference coordinate information, or the tip of the protrusion may be determined by a known method. The specific-shaped part may be, for example, a hole or a protrusion such as a conical shape or a cylindrical shape. For the predetermined calculation method, for example, a hole (hole part) may be detected by the method described later, and the center coordinate information of the hole may be used as the reference coordinate information, or the tip of the protrusion may be determined by a known method. The reference coordinate information may be, but is not limited to, any information that is uniquely determined by calculation. Any shape can be considered as a part of a specific shape. The quasi-point coordinate information is stored in the reference point coordinate information storage unit 122 .

[0033] The specific shape part detection unit 104 detects the reference three-dimensional point group data in response to a predetermined operation by the user, for example. A part of a specific shape in the image sensor 41 is detected, and reference coordinate information is obtained from the part. For example, as shown in FIG. 5, the terminal 1 stores three-dimensional point group data 40 (reference point group A three-dimensional point cloud data41) of an object is used to measure the area around a hole on a given plane. The coordinate information of the points P1-P3 is obtained, and the hole center coordinate information P is calculated from the coordinate information of each of the three points. 11 As a more specific example, the characteristic shape portion detection unit 104 calculates, for example, The coordinates of the center of the provisional circle are calculated by using the perpendicular bisector from the coordinate information of each of the three points P1-P3 around the circle. Calculate the information, and set a cylindrical search range of a specified height from the circle passing through the three points and the temporary circle center coordinate information. Then, a known edge detection method is applied to the 3D point cloud data within the search range. Then, based on the contour coordinate information, a known calculation (e.g. , least squares method of a circle, etc.) to obtain the hole center coordinate information P 11 Calculate.

[0034] The reference search range setting unit 105 selects other measurement object model data based on the reference point coordinate information. 51 (e.g., other three-dimensional point cloud data) For example, in the specific shape part detection unit 104, If a cylindrical search range has already been generated, set it as the reference search range. It is also possible to set a reference search range for an arbitrary three-dimensional shape (solid shape) such as a sphere, cube, rectangular parallelepiped, prism, cylinder, or ellipsoid centered on the reference point coordinate information.

[0035] Based on the reference search range set for other measurement object model data 51 (for example, other three-dimensional point cloud data), the corresponding point coordinate information acquisition unit 106 acquires the corresponding point coordinate information in the measurement object model data 51. The acquired corresponding point coordinate information is stored in the corresponding point coordinate information storage unit 123.

[0036] <Flowchart of the information processing method> FIG. 6 is an example of a flowchart of an information processing method in the information processing system 100 according to the first embodiment.

[0037] First, based on the control by the three-dimensional point cloud data acquisition unit 101 of the terminal 1, the user acquires, by means of the sensor 23, for example, the reference three-dimensional point cloud data 41 of the reference object located on the workbench (SQ101).

[0038] Next, as shown in FIG. 7, the reference three-dimensional point cloud data 41 acquired by the three-dimensional point cloud data display unit 102 is displayed on the terminal 1 (SQ102).

[0039] Next, based on the reference three-dimensional point cloud data 41 by the reference point coordinate acquisition unit 103, for example, an arbitrary number of reference point coordinate information is acquired according to a predetermined operation of the user (SQ103). At this time, as shown in FIG. 8, the specific shape part detection unit 104 detects, for example, a part (hole) of a specific shape in the reference three-dimensional point cloud data 41 according to a predetermined operation of the user, and it is also possible to acquire the center coordinates of each of the holes A - C, which are the reference coordinate information, from the part. ​​​​​​​​​​​​

[0040] Here, next, based on the control by the three-dimensional point cloud data acquisition unit 101 of the terminal 1, , the sensor 23 acquires, for example, three-dimensional point cloud data 51 of another measurement object (however, an object having the same or substantially the same shape, approximation, similarity, same type, same model number, etc. as the reference object) located on the workbench. (SQ104).

[0041] Next, based on the reference point coordinate information, the reference search range setting unit 105 sets a reference search range for acquiring corresponding point coordinate information corresponding to the reference point coordinate information in other measurement object model data 51 (for example, other three-dimensional point cloud data). At this time, for example, the cylindrical search range generated by the specific shape part detection unit 104 is set as the reference search range. (SQ105).

[0042] Next, as shown in FIG. 9, based on the reference search range set for other three-dimensional point cloud data, the corresponding point coordinate acquisition unit 106 acquires the center coordinates of each of the holes A'-C', which are the corresponding point coordinate information, in the other three-dimensional point cloud data. (SQ106).

[0043] As a result, in the reference three-dimensional point cloud data and other three-dimensional point cloud data, three pieces of reference point coordinate information and corresponding point coordinate information based on the same specific shape part are acquired. As described above, the coordinate information of each of these three points can be, for example, reference origin coordinate information with the origin as the origin, X reference point coordinate information with a base point in the X direction from the origin, and Y reference point coordinate information with a base point in the Y direction from the origin. Therefore, using these, a sensor coordinate system (XYZ coordinate system) with the same position as the origin can be set for each object. (SQ107).

[0044] Therefore, the information processing system 100 of the first embodiment applies the reference search range based on the reference point coordinate information for setting the sensor coordinate system (XYZ coordinate system) in the reference three-dimensional model data (particularly the reference three-dimensional point cloud data) to other three-dimensional model data, instead of the fitting by the association between the conventional three-dimensional model data As a result, the corresponding point coordinate information is derived by the same rule. As shown in FIG. 10, when the sensor coordinate systems with the same position as the origin with respect to the object are set for both, it is substantially equivalent to the fitting of both for the object. Thus, the information processing system 100 of the first embodiment sets the sensor coordinate system with the same position as the origin for the object, so that both are substantially fitted by applying the reference search range based on the reference point coordinate information for setting the sensor coordinate system (XYZ coordinate system) in the reference three-dimensional model data (particularly the reference three-dimensional point cloud data) to other three-dimensional model data, instead of the fitting by the association between the conventional three-dimensional model data As a result, as shown in FIG. 10, when the sensor coordinate systems with the same position as the origin with respect to the object are set for both, it is substantially equivalent to the fitting of both for the object.

[0045] <Details of the Second Embodiment> Since the system configuration and fitting method described in the above-described first embodiment are the same they will not be described. In the second embodiment, an example of the configuration for comparing the shapes of the three-dimensional model data (reference three-dimensional point cloud data) of the reference object and the three-dimensional model data (other three-dimensional point cloud data) of other measurement objects with each other after fitting will be described FIG. 11 is a block diagram illustrating the functions implemented in the terminal 1, and is a block diagram illustrating the functions related to the second embodiment in which the functions shown in FIG. 3 are omitted. In the second embodiment the processor 10 of the terminal 1 further includes a reference point coordinate plane calculation unit 201, a reference plane difference value information acquisition unit 202, a corresponding point coordinate plane calculation unit 203, a corresponding plane difference value information acquisition unit 20

[0046] FIG. 11 is a block diagram illustrating the functions implemented in the terminal 1, and is a block diagram illustrating the functions related to the second embodiment in which the functions shown in FIG. 3 are omitted. In the second embodiment the processor 10 of the terminal 1 further includes a reference point coordinate plane calculation unit 201, a reference plane difference value information acquisition unit 202, a corresponding point coordinate plane calculation unit 203, a corresponding plane difference value information acquisition unit 20 4, a difference value information display unit 205, and a difference value information comparison unit 206. The storage 12 of the terminal 1 further includes a reference difference value information storage unit 221, a corresponding difference value information storage unit 222, and a comparison result information storage unit 223. FIG. 11 is a block diagram illustrating the functions implemented in the terminal 1, and is a block diagram illustrating the functions related to the second embodiment in which the functions shown in FIG. 3 are omitted. In the second embodiment the processor 10 of the terminal 1 further includes a reference point coordinate plane calculation unit 201, a reference plane difference value information acquisition unit 202, a corresponding point coordinate plane calculation unit 203, a corresponding plane difference value information acquisition unit 20

[0047] Based on the acquired reference point coordinate information, the reference point coordinate plane calculation unit 201 calculates a reference plane (XY plane) including the reference point coordinate information by a known calculation method. For example, in a three-dimensional coordinate system, if there is at least three-point reference point coordinate information, the XY plane in the XYZ coordinate system is determined. The range of the reference plane may be, for example, an infinite plane range, or a plane range in which all objects are included based on three-dimensional point cloud data, or a range set by the user according to the coordinate information of the sensor coordinate system that includes a part of the object. The reference plane difference value information acquisition unit 202 sets a reference value (for example, 0) in a predetermined coordinate axis direction (Z-axis direction) on the reference plane, and acquires reference difference value information between the reference object model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane. For example, when the reference plane is the XY plane and the reference value is 0 in the Z-axis direction, the Z coordinate information of each point data becomes the reference difference value information. The acquired reference difference value information is stored in the reference difference value information storage unit 221.

[0048] Based on the acquired corresponding point coordinate information, the corresponding point coordinate plane calculation unit 203 calculates a corresponding plane (XY plane) including the corresponding point coordinate information by a known calculation method. For example, in a three-dimensional coordinate system, if there is at least three-point corresponding point coordinate information, the XY plane in the XYZ coordinate system is determined. The range of the corresponding plane can be set in the same way as the reference plane, and it is desirable that the range is the same as the reference plane, but this is not always the case. The corresponding plane difference value information acquisition unit 204 sets a reference value (for example, 0) in a predetermined coordinate axis direction (Z-axis direction) on the corresponding plane, and acquires reference difference value information between the corresponding object model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane. The acquired reference difference value information is stored in the reference difference value information storage unit 221.

[0049] Based on the acquired corresponding point coordinate information, the corresponding point coordinate plane calculation unit 203 calculates a corresponding plane (XY plane) including the corresponding point coordinate information by a known calculation method. For example, in a three-dimensional coordinate system, if there is at least three-point corresponding point coordinate information, the XY plane in the XYZ coordinate system is determined. The range of the corresponding plane can be set in the same way as the reference plane, and it is desirable that the range is the same as the reference plane, but this is not always the case. The corresponding plane difference value information acquisition unit 204 sets a reference value (for example, 0) in a predetermined coordinate axis direction (Z-axis direction) on the corresponding plane, and acquires reference difference value information between the corresponding object model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane. The range of the corresponding plane can be set in the same way as the reference plane, and it is desirable that the range is the same as the reference plane, but this is not always the case.

[0050] The corresponding plane difference value information acquisition unit 204 sets a reference value (for example, 0) in a predetermined coordinate axis direction (Z-axis direction) on the corresponding plane, and acquires reference difference value information between the corresponding object model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane. ​Set it as the reference value (e.g., 0) in the direction, and measure in a predetermined coordinate axis direction with respect to the corresponding plane Obtain the corresponding difference value information between the specified object model data and the reference value. The corresponding difference value information For example, when the corresponding plane is the XY plane and the reference value is 0 in the Z-axis direction, the Z coordinate information of each point data becomes the corresponding difference value information. Note that the obtained corresponding difference value information is stored in the corresponding difference value information storage section 222

[0051] The difference value information display section 205 displays each difference value information with respect to the reference plane or the corresponding plane on the input / output section 14 of the terminal 1, for example as illustrated in FIG. 14. As a display example, for example, as shown in FIG. 14, the reference plane or the corresponding plane is virtually formed and displayed, and point data having at least one of plus or minus difference value information more than the reference value is displayed It may be like this. Also, at the time of display, for example, the difference value information is divided for each predetermined numerical range and colors are set and displayed for each numerical range (so-called heat map) may also be used .

[0052] The difference value information comparison section 206 compares the reference difference value information and the corresponding difference value information with each other, and calculates the comparison result information. The comparison result information, for example, further calculates the difference between the value indicated by the reference difference value information and the value indicated by the corresponding difference value information in each point data, and obtains the deviation amount information between the two It may be. Further, for example, based on the allowable range information of the deviation amount set by the user, a flag indicating inside or outside the allowable range may be set . Then, a color is set for the flag (for example as shown in FIG. 15, green if it is OK within the allowable range, red if it is NG outside the allowable range, and further red if it is on the plus side and blue if it is on the minus side even outside the allowable range), as illustrated in FIG. 13 ​​In the display shown or the display illustrated in FIG. 14, the color may be displayed in a reflected manner. Note that the calculated comparison result information is stored in the comparison result information storage unit 223.

[0053] <Flowchart of the information processing method> FIG. 12 is an example of a flowchart of an information processing method in the information processing system 100 according to the second embodiment. chart.

[0054] First, based on the acquired reference point coordinate information, the reference point coordinate plane calculation unit 201 calculates a reference plane (XY plane) including the reference point coordinate information (SQ201). For example, as shown in FIG. 13 a plane range including all the objects is set.

[0055] Next, the reference plane difference value information acquisition unit 202 sets a reference value (for example, 0) in a predetermined coordinate axis direction (Z-axis direction) on the reference plane, and calculates the reference difference value between the reference object model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane (for example, the Z coordinate information of each point data) (SQ202). At this time, the difference value information display unit 205 may display the reference difference value information with respect to the reference plane on the input / output unit 14 of the terminal 1, for example, as illustrated in FIG. 14

[0056]

[0057] Next, based on the acquired corresponding point coordinate information, the corresponding point coordinate plane calculation unit 203 calculates a corresponding plane (XY plane) including the corresponding point coordinate information (SQ203). For example, as shown in FIG. 1 3, it is set as the same range as the reference plane.

[0057] Next, the corresponding plane difference value information acquisition unit 204 sets a reference value (for example, 0) in a predetermined coordinate axis direction (Z-axis direction) on the corresponding plane, and in the predetermined coordinate axis direction with respect to the corresponding plane (Z-axis direction) and sets a reference value (for example, 0), and acquires the reference difference value between the reference object model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane The corresponding difference value between the measurement object model data and the reference value in it (for example, the Z coordinate information of each point data) is obtained (SQ204). At this time, the difference value information display unit 205 may display the corresponding difference value information with respect to the corresponding plane on the input / output unit 14 of the terminal 1 as exemplified in FIG. 14, for example. That's okay.

[0058] Next, the difference value information comparison unit 206 compares the reference difference value information and the corresponding difference value information with each other and calculates the comparison result information (SQ205). Further, the difference value information comparison unit 206 obtains, for example, the deviation amount information between the two, and as shown in FIG. 15, the deviation amount information may be compared with the allowable range information and color-coded display may be performed inside and outside the allowable range (SQ205).

[0059] Therefore, in the information processing system 100 of the second embodiment, since the sensor coordinate system with the same position as the origin is set for each object according to the configuration of the first embodiment, for example, the difference can be confirmed with the same plane as the reference position. And further, since it is also possible to easily calculate the deviation amount between the two, the shape of the measurement object can be easily compared with the shape of the reference object. It becomes possible to compare.

[0060] As described above, this embodiment has been described, but the above embodiment is for facilitating the understanding of the present invention and is not for limiting and interpreting the present invention. The present invention can be changed and improved without departing from its gist, and equivalents thereof are also included in the present invention.

Explanation of Signs

[0061] 1 Terminal 2 Work robot 21 Arm 22 Tool ​​​​23 sensors 3 controllers

Claims

1. An information processing method, comprising: acquiring reference three-dimensional model data from a reference object by a sensor; acquiring coordinate information of three or more arbitrary reference points from the reference three-dimensional model data by a reference point coordinate acquisition unit; acquiring corresponding three-dimensional model data from a measurement object by a sensor; setting a three-dimensional reference search range based on the reference point coordinate information by a reference search range setting unit; acquiring corresponding point coordinate information corresponding to the reference point coordinate information within the three-dimensional reference search range from three-dimensional point cloud data constituting the corresponding three-dimensional model data by a corresponding point coordinate information acquisition unit; calculating a reference plane including the reference point coordinate information in the reference three-dimensional model data by a reference point coordinate plane calculation unit; setting a reference value in a predetermined coordinate axis direction on the reference plane, and acquiring reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane by a reference difference value information acquisition unit; An information processing method comprising the above steps.

2. The information processing method according to claim 1, wherein each of the reference three-dimensional model data and the corresponding three-dimensional model data is three-dimensional point cloud data.

3. The information processing method according to any one of claims 1 or 2, wherein the step of acquiring the reference point coordinate information includes detecting a portion of a specific shape in the reference three-dimensional model data by a specific shape portion detection unit, and acquiring coordinate information of the portion as reference point coordinate information.

4. The information processing method according to any one of claims 1 to 3, further comprising calculating a corresponding plane including the corresponding point coordinate information in the corresponding three-dimensional model data by a corresponding point coordinate plane calculation unit; and setting a reference value in a predetermined coordinate axis direction on the corresponding plane, and acquiring corresponding difference value information between the corresponding three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the corresponding plane by a corresponding difference value information acquisition unit.

5. The information processing method according to claim 4, further comprising comparing the reference difference value information and the corresponding difference value information with each other by a difference value information comparison unit, and calculating comparison result information.

6. The information processing method according to claim 5, The information processing method is such that the comparison result information is the deviation amount information of both obtained by further calculating the difference between the value indicated by the reference difference value information and the value indicated by the corresponding difference value information.

7. The information processing method according to claim 6, wherein the difference value information comparison unit further includes a step of comparing the allowable range information of the deviation amount set by the user with the deviation amount information, and displaying at least a part of the corresponding three-dimensional model data in different colors according to whether it is within or outside the allowable range.

8. An information processing system, a three-dimensional model data acquisition unit that acquires reference three-dimensional model data and corresponding three-dimensional model data from a reference object and a measurement object by a sensor; a reference point coordinate acquisition unit that acquires reference point coordinate information of any number of three or more from the reference three-dimensional model data; a corresponding point coordinate information acquisition unit that acquires corresponding point coordinate information corresponding to the reference point coordinate information from the three-dimensional point cloud data constituting the corresponding three-dimensional model data within a three-dimensional reference search range; a reference point coordinate plane calculation unit that calculates a reference plane including the reference point coordinate information in the reference three-dimensional model data; a reference difference value information acquisition unit that sets a reference value in a predetermined coordinate axis direction on the reference plane and acquires reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane; An information processing system including the above.

9. A program for causing a computer to execute an information processing method, wherein the program, as the information processing method, includes a step of acquiring reference three-dimensional model data from a reference object by a sensor; a step of acquiring reference point coordinate information of any number of three or more from the reference three-dimensional model data by a reference point coordinate acquisition unit; a step of acquiring corresponding three-dimensional model data from a measurement object by a sensor; a step of setting a three-dimensional reference search range based on the reference point coordinate information by a reference search range setting unit; a step of acquiring corresponding point coordinate information corresponding to the reference point coordinate information from the three-dimensional point cloud data constituting the corresponding three-dimensional model data within the three-dimensional reference search range by a corresponding point coordinate information acquisition unit; a step of calculating a reference plane including the reference point coordinate information in the reference three-dimensional model data by a reference point coordinate plane calculation unit; A step of obtaining reference difference value information between the reference three-dimensional model data and the reference value in the predetermined coordinate axis direction with respect to the reference plane, which is set as a reference value in the predetermined coordinate axis direction on the reference plane by the reference difference value information acquisition unit; A program that causes a computer to execute the above.

Images