Robot arm control system, robot arm control method, and robot arm

Abstract

This enables the proper placement of construction materials in the target area. [Solution] The robot arm control system 1 comprises a robot arm 100 for arranging construction materials and a control device 300 for controlling the robot arm 100. The robot arm 100 captures an image of a position determination identifier installed at a predetermined position in the area to which the robot arm 100 will arrange the construction materials. The control device 300 calibrates the coordinate information of the robot arm based on the image of the position determination identifier captured by the robot arm 100. The control device 300 updates information regarding the plan for arranging construction materials that the robot arm will perform in the area to be arranged using the calibrated coordinate information and the acquired three-dimensional data of the area to be arranged.

Description

Technical Field

[0001] The present invention relates to a robot arm control system, a robot arm control method, and a robot arm.

Background Art

[0002] In recent years, in the field of concrete structures, 3DCP (3D Concrete Printing) technology has been advancing and is used for the placement of construction materials such as concrete at various construction sites. For example, prior art is known in which a nozzle portion provided at the tip of a multi-joint robot arm provided on a vehicle composed of a rail vehicle is driven, and a water-curable hardened body (concrete) is extruded to construct a structure (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above-mentioned prior art, there are problems in appropriately placing construction materials in the target area. For example, the prior art is a technique for automatically placing construction materials based on a preset printing plan using a robot arm, but in an actual construction site or the like, there may be unevenness and it may be difficult to accurately install 3DCP equipment at the planned position.

Means for Solving the Problems

[0005] Therefore, in order to solve the above-mentioned problems and achieve the objective, the present invention provides a robot arm control system comprising: a robot arm for arranging construction materials; and a control device for controlling the robot arm, wherein the robot arm captures an image of a position determination identifier installed at a predetermined position in the area to which the robot arm is to arrange the construction materials; the control device calibrates the coordinate information of the robot arm based on the image of the position determination identifier captured by the robot arm; and updates information regarding the planning of the arrangement of construction materials to be performed by the robot arm in the area to be arranged using the calibrated coordinate information and the acquired three-dimensional data of the area to be arranged. [Effects of the Invention]

[0006] The present invention has the effect of enabling the proper placement of construction materials in the target area. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a diagram illustrating the overall arrangement of construction materials according to the embodiment. [Figure 2] Figure 2 shows an example of the configuration of a robot arm control system according to the embodiment. [Figure 3] Figure 3 illustrates the planar slice and non-planar slice according to the embodiment. [Figure 4] Figure 4 is a table diagram showing an example of a print path plan according to the embodiment. [Figure 5] Figure 5 shows an example of constructing a model according to the uneven surface according to the embodiment. [Figure 6] Figure 6 shows an example of the print path update process based on mesh data according to the embodiment. [Figure 7] Figure 7 shows an example of the coordinate information calibration process according to the embodiment. [Figure 8]Figure 8 shows an example of the process for updating the print path based on a position determination identifier according to the embodiment. [Figure 9] Figure 9 is a flowchart showing an example of the processing procedure by the robot arm control system according to this embodiment. [Figure 10] Figure 10 shows an example of a computer that implements a robot arm control system. [Modes for carrying out the invention]

[0008] From here, embodiments (hereinafter referred to as "embodiments") will be described with reference to the drawings. The embodiments of the robot arm control system, robot arm control method, and robot arm according to the present application will be described in detail below with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, in the drawings, the same parts are denoted by the same reference numerals, and redundant explanations are omitted.

[0009] <Introduction> (background) In recent years, 3DCP technology has been used for arranging construction materials at construction sites and other locations. By using 3DCP technology, structures can be constructed more efficiently than before. The following reference technologies are known as examples of technologies utilizing 3DCP.

[0010] For example, there is a known technology that uses a multi-jointed robotic arm mounted on a rail-road vehicle to drive a concrete-extruding nozzle at its tip to construct structures (see Reference 1 listed below).

[0011] In addition, a technology for performing 3D (3 Dimensions) scanning on uneven ground and generating a print path is known (see, for example, Reference 2 below). Also, a technology for implementing the placement of sidewalk blocks on-site is known (see, for example, Reference 3 below). Further, a technology for continuously placing concrete based on a self-propelled 3D printer (crawler type) is known (see, for example, Reference 4 below).

[0012] (Reference 1) Japanese Patent Application Laid-Open No. 2021-028159 (Reference 2): Development and Verification of 3D Printing Technology on Uneven Ground Surfaces, Taisei Corporation, <URL:https: / / www.taisei.co.jp / tact / tr / 2023 / 16 / >, <Search Date: October 24, 2024>[ (Reference 3): Polyuse, First Domestic Success in On-Site Printing in Public Civil Engineering Works Using a Construction 3D Printer, Polyuse Co., Ltd., <URL:https: / / prtimes.jp / main / html / rd / p / 000000015.000049711.html>, <Search Date: October 24, 2024>[ (Reference 4) Japanese Patent Application Laid-Open No. 2020-172838

[0013] However, there are problems in constructing structures using 3DCP technology. For example, in the case of a gantry-type 3D printer using 3DCP technology, assembling a frame etc. is required, resulting in a decrease in mobility. Also, for a gantry-type 3D printer, since space for installing a frame etc. is required, it may be difficult to place it in a narrow location.

[0014] Therefore, by using a robot arm-type 3D printer, placement in a narrow location becomes possible. However, even in the case of a robot arm-type 3D printer, it is difficult to accurately install the installation position of the robot arm-type 3D printer at the planned position due to restrictions on the accuracy of the installation coordinates of the robot arm and the presence of "uneven ground".

[0015] Here, the above-mentioned "issues regarding the accuracy of installation coordinates" and "issues regarding coping with uneven ground" will be further explained. The "issues regarding the accuracy of installation coordinates" are issues regarding how to align the coordinates in the modeling software with the coordinates at the site. Specifically, unless the origin in the modeling software is accurately aligned with the origin at the site, the placement cannot be carried out at the expected position. Therefore, conventionally, it has been necessary to repeatedly perform manual position adjustment of the robotic arm and dry testing, which may require a significant amount of time before starting printing.

[0016] For example, when constructing a structure based on 3DCP, it is carried out in the procedures of "(1) on-site survey", "(2) installation of the 3D printer", "(3) adjustment of the printing position", and "(4) printing". However, in the "installation of the 3D printer" and "adjustment of the printing position", since the installation position needs to be adjusted in units of several millimeters, a lot of time and labor are required for the preliminary preparation until the start of placement. Also, as described above, high-precision adjustment is required, so it is difficult to efficiently perform continuous construction.

[0017] On the other hand, the "issues regarding coping with uneven ground" are issues regarding coping with the "unevenness" existing on the printing surface. In a construction site where there is an uneven ground, this issue becomes particularly prominent when constructing on-site with unevenness and when printing so that a plurality of structures are connected over a long extension, making it difficult to accurately construct the structure.

[0018] And regarding the above-mentioned issues, it is difficult to cope even by using the above-mentioned reference technologies. For example, in the above-mentioned References 1 and 4, it is difficult to cope with the position control of the machine and the unevenness of the printing location, etc., and on-site construction has not been considered, so it is difficult to solve the above-mentioned issues. Also, References 2 and 3 are technologies premised on using a gantry-type 3D printer, so a vast space is required and there are issues regarding continuous construction.

[0019] (Overall outline of the processing by the robotic arm control system 1) Therefore, the robot arm control system 1 according to this embodiment calibrates the coordinate information of the robot arm's installation position based on the coordinates of the position determination identifier read by the robot arm, and updates the information related to the planning of the placement of construction materials by the robot arm.

[0020] The "position determination identifier" refers to an identifier such as an AR (Augmented Reality) marker placed at a predetermined position near the area where the robot arm will place construction materials, and may hereafter be referred to as an "AR marker." The "information regarding the planning of the placement of construction materials" refers to the position information of the robot arm used to place construction materials such as concrete and mortar in the target area, as well as information regarding the motion plan, and may hereafter be referred to as the "printing path plan."

[0021] Here, the overall picture of the processing by the robot arm control system 1 will be explained using Figure 1. Figure 1 is a diagram illustrating the overall arrangement of construction materials according to the embodiment. Figure 1 shows the "robot coordinate system," which is the coordinate system indicating the reference point of the robot arm 100, the "site coordinate system," which is the coordinate system indicating the reference point of the site, and the "local coordinate system," which shows the coordinates of the multiple AR markers that have been read.

[0022] As shown in Figure 1, the robot arm 100 of the robot arm control system 1 reads an AR marker (hereinafter sometimes simply referred to as "site AR marker") which has positional relationship information associated with the site coordinate system and is installed at a predetermined position in the area where the robot arm 100 is to place construction materials. The reading of the site AR marker is performed by an imaging device (imaging unit 140) attached to the robot arm 100.

[0023] Next, the control device 300 that controls the robot arm 100 of the robot arm control system 1 compares the coordinate information based on the image of the on-site AR marker acquired by the imaging device (imaging unit 140) with the coordinate information of the AR marker in the on-site coordinate system. Then, the control device 300 of the robot arm control system 1 determines how much the robot arm 100, which is installed in a predetermined position, is deviating from the reference, and calibrates the coordinate information included in the print path plan.

[0024] Through the process described above, the robot arm control system 1 according to this embodiment can accurately and continuously construct structures at target locations in accordance with the terrain and the shape of existing facilities using a robot arm 100 that can be installed at any arbitrary location. In other words, the robot arm control system 1 has the effect of enabling the robot arm 100 to appropriately place construction materials in the target area.

[0025] <Robot Arm Control System 1> From here, the detailed functions of the robot arm control system 1 according to this embodiment will be explained using Figure 2. Figure 2 is a diagram showing an example of the configuration of the robot arm control system 1 according to this embodiment.

[0026] As shown in Figure 2, the robot arm control system 1 includes a robot arm 100, which is a robot arm type 3D printer for placing construction materials; an acquisition device 200 for acquiring three-dimensional data such as point cloud data and voxel data for the area where the robot arm 100 places construction materials; and a control device 300 for controlling the robot arm 100. In this embodiment, the above-mentioned "three-dimensional data such as point cloud data and voxel data" may be referred to as "point cloud data" from now on.

[0027] (Robot arm 100) First, let's describe the robot arm 100. The robot arm 100 is a device that places construction materials such as concrete in a target area. As shown in Figure 2, the robot arm 100 has a communication unit 110, a storage unit 120, a control unit 130, an imaging unit 140, and a printing nozzle drive unit 150.

[0028] Furthermore, the robot arm 100 may have an input unit (not shown) such as a touch panel or keyboard for receiving information input from the outside, and a display unit (not shown) such as a printer or display for displaying the operation results of the robot arm 100 or image data acquired by the imaging unit 140.

[0029] (Communications Department 110) The communication unit 110 handles communication related to the input of the print path plan transmitted from the control device 300, and the output of image data of local AR markers captured by the imaging unit 140 and information related to the driving of the print nozzles by the print nozzle drive unit 150. The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). For example, the communication unit 110 controls communication related to various types of information exchanged with connected devices.

[0030] (Storage unit 120) The memory unit 120 is implemented by a storage device such as RAM (Random Access Memory) or a hard disk. The memory unit 120 stores data and programs necessary for various processes performed by the control unit 130.

[0031] For example, the storage unit 120 temporarily or permanently stores image data captured by the imaging unit 140, as well as the drive history and coordinate information of the printing nozzle drive unit 150. The storage unit 120 may also be implemented by a storage system installed outside the robot arm 100.

[0032] (Control unit 130) The control unit 130 is implemented using a CPU (Central Processing Unit) or the like, and executes processing programs stored in the storage unit 120. The control unit 130 uses the information stored in the storage unit 120 to control the processing of the imaging unit 140 and the printing nozzle drive unit 150, which will be described later. The control unit 130 also has a transmission unit 131.

[0033] (Transmitter 131) The transmitting unit 131 acquires image data captured by the imaging unit 140 from the storage unit 120 and transmits it to the control device 300 via the communication unit 110 described above. The transmission of image data by the transmitting unit 131 may be performed when predetermined conditions are met, such as when a command is issued by an administrator, when a predetermined date and time arrives, or when the amount of stored image data exceeds a threshold.

[0034] (Imaging unit 140) The imaging unit 140 captures images of the local AR markers based on an imaging device such as a camera attached to a predetermined position on the robot arm 100. The imaging unit 140 then stores the captured image data of the local AR markers in the storage unit 120.

[0035] (Print nozzle drive unit 150) The printing nozzle drive unit 150 drives the printing nozzles to place construction materials at predetermined locations using a printing path plan updated based on coordinate information calibrated using images of on-site AR markers captured by the imaging unit 140. The printing nozzle drive unit 150 then places the construction materials in the target area based on the printing path plan.

[0036] In this embodiment, the printing nozzle is a nozzle attached to the tip of the robot arm 100 for positioning construction materials. The printing nozzle drive unit 150 controls the robot arm 100 based on information of six degrees of freedom: the X, Y, and Z axes which define the position of the printing nozzle, and the rotations Rx, Ry, and Rz around each axis which define the posture of the printing nozzle, thereby positioning the construction materials on an uneven surface based on non-planar slicing.

[0037] Here, the "planar slice" and "non-planar slice" described above will be explained using Figure 3. Figure 3 is a diagram illustrating the planar slice and non-planar slice according to the embodiment. Figure 3 shows a planar slice (Figure 3(1)) and a non-planar slice (Figure 3(2)).

[0038] The planar slice shown in Figure 3(1) is a method of representing the slope of a structure by making the height of each layer uniform when stacking and arranging construction materials and adjusting the arrangement area (Figure 3(1-1)). On the other hand, the non-planar slice shown in Figure 3(2) is a method of representing a continuous slope without interruption by controlling the height of each layer when stacking and arranging construction materials (Figure 3(2-1)).

[0039] In this embodiment, the printing nozzle drive unit 150, based on a 6-axis controlled printing nozzle, arranges construction materials on an uneven surface by combining "planar slicing" and "non-planar slicing".

[0040] (Acquisition device 200) Returning to Figure 2, the acquisition device 200 will now be described. The acquisition device 200 is a device that acquires point cloud data of the area where construction materials are placed by the robot arm 100. As shown in Figure 2, the acquisition device 200 has a communication unit 210, a storage unit 220, and a control unit 230.

[0041] Furthermore, the acquisition device 200 may have an input unit (not shown) such as a touch panel or keyboard for receiving information input from an external source, and a display unit (not shown) such as a printer or display for displaying the acquired point cloud data.

[0042] (Communications Section 210) The communication unit 210 handles communication related to the input of information such as commands for acquiring point cloud data, and the output of acquired point cloud data. The communication unit 210 is implemented, for example, by a NIC (Network Interface Card). For example, the communication unit 210 controls communication related to various types of information exchanged with connected devices.

[0043] (Storage unit 220) The memory unit 220 is implemented by a storage device such as RAM or a hard disk. The memory unit 220 stores data and programs necessary for various processes performed by the control unit 230. For example, the memory unit 220 temporarily or permanently stores point cloud data acquired by the acquisition unit 231, which will be described later. The memory unit 120 may be implemented by a storage system installed outside the acquisition device 200.

[0044] (Control unit 230) The control unit 230 is implemented using a CPU or the like and executes the processing program stored in the memory unit 220. Specifically, the control unit 230 has an acquisition unit 231 and a transmission unit 232.

[0045] (Acquisition part 231) The acquisition unit 231 acquires point cloud data of the target area. For example, based on known technologies such as 3D laser scanning technology, the acquisition unit 231 scans the planned placement locations of construction materials (the target area) and acquires point cloud data related to the target area as point cloud data. The acquisition unit 231 then stores the acquired point cloud data, such as the point cloud data, in the storage unit 220.

[0046] (Transmitter 232) The transmitting unit 232 retrieves the point cloud data acquired by the acquisition unit 231 from the storage unit 220 and transmits it to the control device 300 via the communication unit 210 described above. The transmission of point cloud data by the transmitting unit 232 may be performed when predetermined conditions are met, such as when a command is issued by an administrator, when a predetermined date and time arrives, or when the amount of accumulated point cloud data exceeds a threshold.

[0047] (Control device 300) Next, the control device 300 will be described. The control device 300 is a device that controls the operation of the robot arm 100 by performing actions such as calibrating the coordinate information of the robot arm 100 and updating the print path plan. As shown in Figure 2, the control device 300 has a communication unit 310, a storage unit 320, and a control unit 330.

[0048] Furthermore, the control device 300 may have an input unit (not shown) such as a touch panel or keyboard for receiving information input from the outside, and a display unit (not shown) such as a printer or display for displaying the updated print path plan.

[0049] (Communications Department 310) The communication unit 310 handles communication related to the input of image data of local AR markers transmitted by the robot arm 100, point cloud data transmitted by the acquisition device 200, and the output of a print path plan to the robot arm 100. The communication unit 310 is implemented, for example, by a NIC. For example, the communication unit 310 controls communication related to various types of information exchanged with connected devices.

[0050] (Storage unit 320) The memory unit 320 is implemented by a storage device such as RAM or a hard disk. The memory unit 320 stores data and programs necessary for various processes performed by the control unit 330. Specifically, the memory unit 320 has a print path planning memory unit 321.

[0051] Furthermore, the storage unit 320 stores image data of local AR markers captured by the robot arm 100 (imaging unit 140), point cloud data relating to the target area acquired by the acquisition device 200, and the like. The storage unit 320 may also be implemented by a storage system installed outside the control device 300.

[0052] (Print path planning storage unit 321) The print path planning storage unit 321 stores the print path plan used by the robot arm 100 when placing construction materials in a target area. The print path planning storage unit 321 stores information identifying the area to which construction materials are to be placed, coordinate information relating to the target area and the installation position of the robot arm 100, and the placement plan of construction materials in the target area.

[0053] Here, an example of a print path plan stored in the print path plan storage unit 321 will be explained using Figure 4. Figure 4 is a table diagram showing an example of a print path plan according to the embodiment.

[0054] For example, the print path plan storage unit 321 stores the "target area," "coordinate information," and "layout plan" in a table format, as shown in Figure 4, associated with "No," which is information that identifies the data of individual print path plans. The letters "A to C" shown in the table diagram of Figure 4 are information that indicates a legend for the information stored for each item, and may be a combination of one or more of the following: text, numbers, formulas, diagrams, etc.

[0055] "Target area" refers to information that identifies the area where construction materials are to be placed, and includes, for example, information that combines text, characters, symbols, etc. "Coordinate information" refers to predetermined coordinate information related to the target area, and includes, for example, the coordinates of the AR marker in the site coordinate system, the coordinates of the placement positions of the construction materials, the coordinates of the installation position of the robot arm 100 in the robot coordinate system, and the coordinates of the on-site AR marker in the local coordinate system. "Placement plan" refers to a printing route plan, and includes, for example, information such as the type of construction material, placement position, printing route (placement order), and placement quantity.

[0056] (Control unit 330) Returning to Figure 2, let's continue the explanation. The control unit 330 is implemented using a CPU, NP (Network Processor), FPGA (Field Programmable Gate Array), etc., and executes the processing program stored in the memory unit 320. Specifically, the control unit 330 has a reception unit 331, an update unit 332, and a transmission unit 333.

[0057] (Reception desk 331) The reception unit 331 receives image data of on-site AR markers captured by the robot arm 100 (imaging unit 140), point cloud data relating to the target area acquired by the acquisition device 200, etc. The reception unit 331 then stores the received image data and point cloud data in the storage unit 320.

[0058] (Update section 332) The update unit 332 calibrates the coordinate information of the robot arm 100 based on the image data of the on-site AR marker captured by the imaging unit 140 of the robot arm 100. Then, the update unit 332 updates the printing path plan related to the placement of construction materials that the robot arm 100 will execute in the target area, using the calibrated coordinate information and the point cloud data of the target area acquired by the acquisition device 200.

[0059] Specifically, the update unit 332 updates the print path plan using "mesh data constructed using point cloud data acquired by the acquisition device 200" and "image data of on-site AR markers captured by the robot arm 100".

[0060] (An example of mesh data construction by update unit 332) From here, we will explain in more detail how the update unit 332 constructs the mesh data. First, we will explain the process of constructing the mesh data using the acquired point cloud data and the control of the robot arm 100 using Figure 5.

[0061] Figure 5 shows an example of model construction according to an uneven surface according to the embodiment. Figure 5 shows a robot arm 100 that places construction materials along an updated print path plan based on the constructed mesh data, an acquisition device 200 that acquires point cloud data related to the area to be placed, and a control device 300 that constructs the mesh data and updates the print path plan.

[0062] In the example shown in Figure 5, the control device 300 (update unit) uses point cloud data acquired by the acquisition device 200 to construct mesh data of the locations where construction materials will be placed in the three-dimensional space related to the target area. Then, the control device 300 (update unit) updates the print path plan using the constructed mesh data and image data of on-site AR markers captured by the robot arm 100.

[0063] As shown in Figure 5(1), when the objective is to improve work efficiency by directly placing construction materials into gaps in the placement base or slope, it is difficult to place the structure according to the drawing if there are irregularities (uneven surfaces) on the placement surface. Therefore, an acquisition device 200 such as a laser scanner is used to scan the uneven surface of the site in advance and update the printing path plan of the robot arm 100.

[0064] First, the acquisition device 200 (acquisition unit) acquires point cloud data about the planned placement locations of construction materials (Figure 5 (2-1)). Then, the acquisition device 200 (transmission unit) transmits the acquired point cloud data to the control device 300 (Figure 5 (2-2)).

[0065] The control device 300 (update unit) uses the point cloud data transmitted from the acquisition device 200 to construct mesh data corresponding to the uneven surfaces. In the example shown in Figure 5, there are two uneven surfaces: the "back surface (Figure 5 (3-1))" and the "bottom surface (Figure 5 (3-2))". Therefore, the control device 300 (update unit) constructs mesh data using a slicing method of the model corresponding to each uneven surface (Figure 5 (3)).

[0066] For example, the control device 300 (update unit) constructs mesh data for the "back surface (3-1 in Figure 5)" using the "planar slice" explained using (1) in Figure 3. On the other hand, the control device 300 (update unit) constructs mesh data for the "bottom surface (3-2 in Figure 5)" using the "non-planar slice" explained using (2) in Figure 3.

[0067] Then, the robot arm 100 (print nozzle drive unit) places construction materials in the target area (Figure 5 (4)) based on the updated print path plan using the constructed mesh data.

[0068] Here, the construction of mesh data by the update unit 332 and the updating of the print path plan using the constructed mesh data will be explained with reference to Figure 6. Figure 6 is a diagram showing an example of the print path update process based on mesh data according to the embodiment.

[0069] First, the acquisition device 200 (acquisition unit) acquires point cloud data for the area where construction materials are to be placed (Figure 6 (1)). At this time, the update unit 332 can update the print path plan by constructing in three-dimensional space, together with mesh data, on-site AR markers placed at predetermined positions related to the target area included in the acquired point cloud data.

[0070] Next, the update unit 332 converts the acquired point cloud data into mesh data relating to the target region (Figure 6(2)). In this embodiment, the update unit 332 can construct mesh data (convert point cloud data to mesh data) based on known techniques. Furthermore, the mesh data shown in Figure 6(2) is a simplified example and is not limited to this.

[0071] The update unit 332 constructs a structural body (shown as "A" in Figure 6(3)) built by the robot arm 100 and a structural interpolation part (shown as "B" in Figure 6(3)) that complements the relationship between the structural body and the mesh data (Figure 6(3)).

[0072] The update unit 332 connects the lines of the constructed structure body and the structural complement parts and converts them into slice data (Figure 6 (4)). Then, the update unit 332 applies the converted slice data to the print path and updates the print path plan.

[0073] (Update of coordinate information by update unit 332) Next, the coordinate information update process by the update unit 332 will be explained using Figure 7. Figure 7 is a diagram showing an example of the coordinate information calibration process according to the embodiment.

[0074] First, the pre-calibration of the robot arm 100 will be explained. As shown in Figure 7(1), the robot arm 100 is controlled based on six values ​​(Tool X to Tool Rz). Specifically, Tool X, Y, and Z represent the position, and Tool Rx, Ry, and Rz represent the orientation. In this embodiment, the initial values ​​of the six values ​​of the robot arm 100 can be calibrated by estimating the above six values ​​based on the captured on-site AR markers to understand the robot's position and orientation.

[0075] Next, the update unit 332 calibrates the coordinate information of the robot arm 100 as the zero point in the three-dimensional space constructed for the target area, after matching the pixels related to the local AR marker (first identifier), which is a pre-registered position determination identifier, with the pixels related to the AR marker (second identifier) ​​(hereinafter sometimes referred to as "local coordinate system AR marker") included in the image of the position determination identifier captured by the robot arm 100.

[0076] Specifically, the update unit 332 compares the acquired on-site AR markers (for example, (2-1) to (2-3) in Figure 7) with pre-registered on-site coordinate system AR markers and performs alignment until the pixels of the two AR markers match. In addition, the update unit 332 can calibrate the position information of the main structure to be constructed based on the position information of the on-site AR markers (for example, (2-1) to (2-3) in Figure 7) ((2-4) in Figure 7).

[0077] The update unit 332 then applies the coordinate information at the time of alignment completion to the three-dimensional space and updates the print path plan.

[0078] Here, an example of the calibration of coordinate information by the update unit 332 and the updating of the print path plan will be explained using Figure 8. Figure 8 is a diagram showing an example of the print path update process based on the position determination identifier according to the embodiment.

[0079] First, the update unit 332 associates the installation point of the robot arm 100 in the three-dimensional space constructed for the target area (robot coordinate system) with the origin in the three-dimensional space (site coordinate system) (Figure 8 (1)).

[0080] Next, the update unit 332 acquires coordinate information of the printing nozzle tip of the robot arm 100 based on the captured second identifier (Figure 8(2)). At this time, as shown in Figure 8(2-1), the relative position between the printing nozzle and the camera is set in advance.

[0081] Next, the update unit 332 uses the acquired coordinate information of the printing nozzle tip to reposition the installation point of the robot arm 100 from the origin in three-dimensional space to the local coordinate system relating to the target area, thereby calibrating the coordinate information of the robot arm 100.

[0082] Specifically, the update unit 332 applies the acquired coordinate information to the three-dimensional spatial data. For example, as shown in (3) of Figure 8, the update unit 332 applies the acquired coordinate information to the three-dimensional spatial data as text data.

[0083] The update unit 332 then sets a local coordinate system consisting of the three local AR markers read in three-dimensional space, and repositions the origin of the local coordinate system from the origin in three-dimensional space (local coordinate system) to reflect the local coordinate system in three-dimensional space (Figure 8 (4)).

[0084] (Transmitter 333) The transmitting unit 333 transmits the print path plan updated by the updating unit 332 to the robot arm 100, which places construction materials in the target area, via the communication unit 310.

[0085] (Processing procedure) From here, the processing procedure of the robot arm control system 1 according to this embodiment will be explained with reference to Figure 9. Figure 9 is a flowchart showing an example of the processing procedure of the robot arm control system 1 according to this embodiment.

[0086] If mesh data is to be constructed (Yes in S101), the acquisition unit 231 of the acquisition device 200 performs point cloud data acquisition (S102). Next, the acquisition device 200 transmits the acquired point cloud data to the control device 300 (S103). Then, the update unit 332 of the control device 300 constructs mesh data using the transmitted point cloud data (S104).

[0087] On the other hand, if mesh data is not constructed (No. S101), the robot arm control system 1 skips steps S102 to S104. Note that "not constructing mesh data" includes cases where mesh data has already been constructed, or where mesh data construction is unnecessary for reasons such as the absence of uneven surfaces.

[0088] The imaging unit 140 of the robot arm 100 captures an image of the local AR marker (S105). The robot arm 100 then transmits the captured image of the local AR marker to the control device 300 (S106).

[0089] The update unit 332 of the control device 300 calibrates the coordinate information of the robot arm 100 using the local AR marker and the local coordinate system AR marker (S107). Next, if mesh data exists, the update unit 332 updates the print path plan using the mesh data and the configured coordinate information (S108). Next, the transmission unit 333 of the control device 300 transmits the print path plan to the robot arm 100 (S109). Then, the robot arm control system 1 completes the process.

[0090] (effect) Next, we will explain the effects of the robot arm control system 1 according to this embodiment. Conventionally, there are challenges in efficiently constructing structures using 3DCP technology.

[0091] Therefore, the robot arm 100 (imaging unit 140) included in the robot arm control system 1 according to this embodiment captures an image of a local AR marker installed at a predetermined position in the area where the robot arm 100 is to place construction materials, based on an imaging device attached to a predetermined position on the robot arm 100.

[0092] The control device 300 (update unit 332) included in the robot arm control system 1 calibrates the coordinate information of the robot arm 100 based on the image of the on-site AR marker captured by the imaging unit 140 of the robot arm 100. Next, the control device 300 (update unit 332) updates the printing path plan related to the placement of construction materials that the robot arm 100 will execute in the target area, using the calibrated coordinate information and the acquired point cloud data of the target area.

[0093] The robot arm 100 (print nozzle drive unit 150) then drives the print nozzle to place construction materials in predetermined positions using a print path plan updated based on coordinate information calibrated using images of on-site AR markers captured by the imaging unit 140.

[0094] Therefore, the robot arm control system 1 of this embodiment facilitates the position control of the robot arm 100, taking into account uneven surfaces, thereby enabling continuous on-site construction. Furthermore, the robot arm control system 1 enables continuous construction using a robot arm 100 that is installed at any location. In other words, the robot arm control system 1 has the effect of enabling the appropriate placement of construction materials in the target area.

[0095] From here, the effects of each functional unit will be explained individually. The control device 300 (update unit 332) calibrates the coordinate information of the robot arm 100 as the zero point in the three-dimensional space constructed for the target area, after comparing the pixels related to the pre-registered on-site coordinate system AR marker with the pixels related to the captured on-site AR marker.

[0096] Specifically, the control device 300 (update unit 332) associates the installation point of the robot arm 100 in the three-dimensional space constructed for the target area with the origin in the three-dimensional space. Next, the control device 300 (update unit 332) acquires coordinate information of the printing nozzle tip of the robot arm 100 based on the captured on-site coordinate system AR marker. Then, using the acquired coordinate information of the printing nozzle tip, the control device 300 (update unit 332) repositions the installation point of the robot arm 100 from the origin in the three-dimensional space to the on-site coordinate system related to the target area, thereby calibrating the coordinate information of the robot arm 100.

[0097] Through the process described above, the robot arm control system 1 can efficiently calibrate the coordinate information of the robot arm 100 by imaging the on-site AR marker. Therefore, by efficiently calibrating the coordinate information of the robot arm 100, the robot arm control system 1 achieves the effect of appropriately placing construction materials in the target area.

[0098] The control device 300 (update unit 332) converts the point cloud data into mesh data relating to the target area. Next, the control device 300 (update unit 332) updates the print path plan by constructing the main structure built by the robot arm 100 and a structural complementation part that complements the gap between the main structure and the mesh data, based on the mesh data relating to the target area.

[0099] Through the process described above, the robot arm control system 1 can update the printing path plan appropriately according to the uneven surface, even if an uneven surface exists in the target area, by constructing mesh data for the target area including the uneven surface.

[0100] Specifically, the robot arm control system 1 constructs a structural model that complements the mesh data relating to the uneven surface of the target area and the model of the structural body, thereby enabling the appropriate placement of construction materials even in a target area that includes uneven surfaces. Therefore, the robot arm control system 1 achieves the effect of appropriately placing construction materials in the target area by efficiently calibrating the coordinate information of the robot arm 100.

[0101] Furthermore, the control device 300 (update unit 332) constructs local AR markers placed at predetermined positions related to the target area included in the acquired point cloud data in three-dimensional space together with the mesh data, and updates the print path plan.

[0102] Through the process described above, the robot arm control system 1 can construct a model of the on-site AR markers based on the point cloud data of the on-site AR markers installed in the target area, while simultaneously constructing the mesh data. The robot arm control system 1 can then use the constructed model of the on-site AR markers to update the print path plan, thereby, for example, reproducing the actual position of the slope in three-dimensional space.

[0103] The robot arm 100 (printing nozzle drive unit 150) is controlled based on information of six degrees of freedom: the X, Y, and Z axes which define the position of the printing nozzle of the robot arm 100, and the rotations Rx, Ry, and Rz around each axis which define the orientation of the printing nozzle of the robot arm 100, thereby positioning construction materials on an uneven surface based on non-planar slicing.

[0104] Through the process described above, the robot arm control system 1 updates the printing path plan using a non-planar slice or a planar slice on an uneven surface, using a 6-axis controlled robot arm 100. As a result, the robot arm control system 1 has the effect of enabling the appropriate placement of construction materials even in areas where uneven surfaces exist.

[0105] <Variation> The following describes modified examples implemented by the robot arm control system 1 according to this embodiment.

[0106] (Data, etc.) The names of the arrangement, construction materials, construction site, site coordinate system, local coordinate system, robot coordinate system, AR markers (local AR markers, site coordinate system AR markers), planar slices, non-planar slices, point cloud data, mesh data, print path planning, robot arm 100 included in the robot arm control system 1, acquisition device 200, and control device 300, etc., used in the description of this embodiment are merely examples and can be changed at will.

[0107] Furthermore, the term "construction materials" as used above includes "materials such as concrete used for the construction of building structures and civil engineering structures (construction materials)." Also, the term "construction site" as used above includes "a site where building structures and civil engineering structures are constructed (construction site)."

[0108] For example, the print path plan storage unit 321 is described as storing "target area," "coordinate information," and "layout plan" in a table format, as shown in Figure 4, associated with "No," which is information that identifies the data of individual print path plans. However, the item names and the information stored associated with those items are not limited.

[0109] (Flowcharts, etc.) The steps in a flowchart may be rearranged as long as they do not contradict each other, and some steps may be omitted. Furthermore, conjunctions such as "next," "continue," "in addition," "at this time," and "on this occasion" in the flowchart description do not limit the order or timing of the processes in the flowchart. Also, the steps, processes, and names of steps and processes in a flowchart are merely examples and may be changed as desired.

[0110] <Hardware Configuration> Furthermore, the various devices included in the robot arm control system 1 according to the above embodiment are realized by a computer 1000 having a configuration such as that shown in Figure 10. Figure 10 is a diagram showing an example of a computer that realizes the robot arm control system 1.

[0111] Computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a central processing unit 1030, memory 1040, storage 1050, output interface 1060, input interface 1070, and communication interface 1080 are connected by a bus 1090.

[0112] The central processing unit 1030 operates based on programs stored in the memory 1040 and storage 1050, as well as programs read from the input device 1020, and executes various processes. The memory 1040 is a memory device, such as RAM, that temporarily stores data used by the central processing unit 1030 for various calculations. The storage 1050 is a storage device where data used by the central processing unit 1030 for various calculations and various databases are registered, and is implemented using ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, etc.

[0113] Output IF1060 is an interface for transmitting information to be output to output devices 1010 that output various types of information, such as monitors and printers. It can be implemented using connectors of standards such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface). Input IF1070 is an interface for receiving information from various input devices 1020, such as mice, keyboards, and cameras. It can be implemented using USB, for example. Input devices 1020 may also be devices that read information from optical recording media, magneto-optical recording media, tape media, magnetic recording media, or semiconductor memory, or external storage media such as USB memory.

[0114] The communication interface 1080 receives data from other devices via the network N and sends it to the central processing unit 1030, and also transmits data generated by the central processing unit 1030 to other devices via the network N. The central processing unit 1030 controls the output device 1010 and input device 1020 via the output IF 1060 and input IF 1070. For example, the central processing unit 1030 loads a program from the input device 1020 or storage 1050 into memory 1040 and executes the loaded program.

[0115] For example, if the computer 1000 functions as an information processing device or the like that constitutes the robot arm control system 1, the central processing unit 1030 of the computer 1000 realizes the functions of the control units of each device by executing a program loaded onto the memory 1040.

[0116] Although some embodiments of the present invention have been described above with reference to the drawings, these are illustrative examples, and the present invention may be implemented in other forms by various modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention. Furthermore, the terms "section, module, unit" used above can be read as "means," "circuit," etc. [Explanation of symbols]

[0117] 1. Robot arm control system 100 robotic arms 110,210,310 Communications Department 120,220,320 storage section 130,230,330 Control Unit 131,232,333 Transmitter 140 Imaging Unit 150 Print nozzle drive unit 200 Acquisition device 231 Acquisition Department 300 Control device 321 Printing path planning memory unit 331 Reception Department 332 Update Department

Claims

1. A robot arm control system comprising a robot arm for arranging construction materials and a control device for controlling the robot arm, The robotic arm is The robot arm captures an image of a position determination identifier installed at a predetermined position in the area where the construction materials are to be placed. The control device is Based on the image of the position determination identifier captured by the robot arm, the coordinate information of the robot arm is calibrated. Using the calibrated coordinate information and the acquired three-dimensional data of the target area, the robot arm updates information regarding the planning of the placement of construction materials in the target area. A robot arm control system characterized by the following features.

2. The control device is A pixel relating to a first identifier, which is a pre-registered location determination identifier, The coordinate information of the robot arm, when matched with the pixels corresponding to the second identifier contained in the image of the position determination identifier that was captured, is calibrated as the zero point in the three-dimensional space constructed for the target region. The robot arm control system according to feature 1.

3. The control device is The installation point of the robot arm in the three-dimensional space constructed for the target region is made to correspond with the origin in the three-dimensional space. Based on the imaged second identifier, coordinate information of the printing nozzle tip of the robot arm is obtained. Using the acquired coordinate information of the printing nozzle tip, the installation point of the robot arm is reset from the origin in the three-dimensional space to the local coordinate system relating to the target area, and the coordinate information of the robot arm is calibrated. The robot arm control system according to claim 2.

4. The control device is The three-dimensional point cloud data is converted into mesh data relating to the target region. With respect to the mesh data relating to the target region, a structural body constructed by the robot arm and a structural complementation portion that complements the relationship between the structural body and the mesh data are constructed. To update information regarding the planning of the placement of the aforementioned construction materials, The robot arm control system according to feature 1.

5. The control device is The position determination identifiers, which are placed at predetermined positions in the target region included in the acquired point cloud data, are constructed in three-dimensional space together with the mesh data. To update information regarding the planning of the placement of the aforementioned construction materials, The robot arm control system according to feature 4.

6. The robotic arm is The robot arm is controlled based on information of six degrees of freedom: the X, Y, and Z axes that define the position of the printing nozzle, and the rotations Rx, Ry, and Rz around each axis that define the orientation of the printing nozzle, and the construction materials are positioned on an uneven surface based on non-planar slicing. A robot arm control system according to any one of claims 1 to 5.

7. A robot arm control method implemented by a robot arm control system having a robot arm for arranging construction materials and a control device for controlling the robot arm, The robot arm, The robot arm captures an image of a position determination identifier installed at a predetermined position in the area where the construction materials are to be placed. The control device, Based on the image of the position determination identifier captured by the robot arm, the coordinate information of the robot arm is calibrated. Using the calibrated coordinate information and the acquired three-dimensional data of the target area, the robot arm updates information regarding the planning of the placement of construction materials in the target area. A robot arm control method characterized by the following:

8. An imaging unit captures an image of a position determination identifier installed at a predetermined position in the area where the robot arm is to place construction materials, based on an imaging device attached to a predetermined position on the robot arm. A printing nozzle drive unit drives a printing nozzle to position the construction materials at a predetermined location using information regarding the planning of the placement of construction materials, which has been updated based on coordinate information calibrated using the image of the position determination identifier captured by the imaging unit. A robotic arm characterized by having the following features.

Images