Automated Installation Method of Building Modules Based on Robotic Arm Collaboration

Abstract

This invention provides an automated installation method for building modules based on robotic arm collaboration, belonging to the field of modular building technology. The method includes: controlling a hoisting device to move a module to be installed to an adjustable position; the module to be installed is connected to an auxiliary frame, with the auxiliary frame positioned above the module; a robotic arm is fixed to the auxiliary frame; acquiring a target image; determining the alignment result between the installation point on the module to be installed and the target installation position point based on the target image; the target image being obtained by imaging the module to be installed and the target installation position point; controlling the hoisting device to move the module to be installed to the target position based on the alignment result, so that the installation point on the module to be installed aligns with the target installation position point; and controlling the robotic arm on the auxiliary frame to retrieve the end effector corresponding to the module to be installed and the installation materials from within the auxiliary frame, enabling the robotic arm to install the module. This invention can achieve automated installation of building modules, reducing labor costs.

Description

Technical Field

[0001] This invention relates to the field of modular building technology, and in particular to an automated installation method for building modules based on robotic arm collaboration. Background Technology

[0002] Modular construction involves producing structural modules of a building in a different location and then assembling them on-site to complete the construction of the building. In the current technology, when installing building modules, the building modules are first hoisted to the installation site, and then more than 10 workers are usually required to perform the operations of straightening and installation (including grouting, bolt fixing and connection, etc.), which wastes a lot of manpower. Summary of the Invention

[0003] This invention provides an automated installation method for building modules based on robotic arm collaboration, which solves the problem that modular buildings in the prior art require a large amount of manual labor and realizes the automated installation of building modules.

[0004] This invention provides an automated installation method for building modules based on robotic arm collaboration, comprising:

[0005] The hoisting equipment is controlled to move the module to be installed to the adjustment position. The distance between the adjustment position and the target installation position of the module to be installed is not less than a distance threshold. The module to be installed is connected to the auxiliary frame and the auxiliary frame is above the module to be installed. At least one robotic arm is fixed on the auxiliary frame.

[0006] A target image is acquired, and the alignment result between the mounting point on the module to be installed and the target mounting position point is determined based on the target image. The alignment result reflects the difference between the mounting point on the module to be installed and the target mounting position point. The target image is obtained by imaging the module to be installed and the target mounting position point.

[0007] Based on the alignment result, the hoisting equipment is controlled to move the module to be installed to the target position so that the installation point on the module to be installed is aligned with the target installation position point;

[0008] The robotic arm on the auxiliary frame is controlled to retrieve the end effector and installation materials corresponding to the module to be installed from within the auxiliary frame, so that the robotic arm can install the module to be installed.

[0009] According to the present invention, an automated installation method for building modules based on robotic arm collaboration is provided, wherein determining the alignment result between the installation point on the module to be installed and the target installation position point based on the target image includes:

[0010] The target image is input into a trained image processing model to obtain the installation point detection result output by the image processing model. The installation point detection result includes the installation point and the target installation location point identified from the target image.

[0011] The alignment result is generated based on the installation point detection result, and the alignment result includes the movement direction of the module to be installed.

[0012] According to the present invention, an automated installation method for building modules based on robotic arm collaboration is provided, wherein controlling the hoisting equipment to move the module to be installed to the target position based on the alignment result includes:

[0013] The hoisting equipment is controlled to move the module to be installed in the direction of movement according to the alignment result with a first moving step.

[0014] The hoisting equipment is controlled to move the module to be installed vertically downwards by a preset second moving step.

[0015] Update the target image, update the installation point detection result based on the updated target image, and update the movement direction and the first movement step size based on the updated installation point detection result;

[0016] The step of controlling the hoisting equipment to move the module to be installed in the direction of movement in the alignment result with a first step size is re-executed until the installation point in the installation point detection result coincides with the target installation position point.

[0017] Control the hoisting equipment to move the module to be installed vertically so that the installation module reaches the target position.

[0018] According to the present invention, an automated installation method for building modules based on robotic arm collaboration is provided, wherein before controlling the hoisting equipment to move the module to be installed in the direction of movement according to the alignment result by a first step, the method further includes:

[0019] Obtain the horizontal information sent by the horizontal sensor in the robotic arm;

[0020] The hoisting equipment is controlled to rotate the module to be installed based on the horizontal information.

[0021] According to the present invention, an automated installation method for building modules based on robotic arm collaboration is provided. The end effector within the auxiliary frame includes at least one of a filling actuator, a cutting actuator, a cleaning actuator, and a fastening actuator. The filling actuator is used to fill building materials into the connection between the module to be installed and the building foundation. The cutting actuator is used to cut excess building materials from the module to be installed. The cleaning actuator is used to clean the connection between the module to be installed and the building foundation. The fastening actuator is used to fasten the fasteners in the module to be installed.

[0022] According to the present invention, an automated installation method for building modules based on robotic arm collaboration is provided, wherein controlling the robotic arm on the auxiliary frame to obtain the end effector corresponding to the module to be installed and the installation materials from the auxiliary frame so as to install the module by the robotic arm, includes:

[0023] Obtain the building module category of the module to be installed, and determine the end effector information and installation material information corresponding to the module to be installed based on the building module category;

[0024] Based on the end effector information, the robotic arm is controlled to retrieve the end effector corresponding to the module to be installed from the auxiliary frame and connect it to the robotic arm. Based on the installation material information, the robotic arm is controlled to retrieve the installation material from the auxiliary frame.

[0025] Acquire an installation image, and acquire the installation point identification result in the installation image. The installation point identification result includes the position of the installation point of the module to be installed in the installation image. The installation image is an image obtained by taking a picture of the module to be installed after it has been moved to the target position.

[0026] Based on the installation point identification result, the robotic arm is controlled to install the module to be installed.

[0027] The present invention also provides an automated installation device for building modules based on robotic arm collaboration, comprising:

[0028] A coarse movement module is used to control the hoisting equipment to move the module to be installed to an adjustment position. The distance between the adjustment position and the target installation position of the module to be installed is not less than a distance threshold. The module to be installed is connected to an auxiliary frame, and the auxiliary frame is above the module to be installed. At least one robotic arm is fixed on the auxiliary frame.

[0029] An image processing module is used to acquire a target image and determine the alignment result between the mounting point on the module to be installed and the target mounting position point based on the target image. The alignment result reflects the difference between the mounting point on the module to be installed and the target mounting position point. The target image is obtained by imaging the module to be installed and the target mounting position point.

[0030] A fine movement module is used to control the hoisting equipment to move the module to be installed to the target position based on the alignment result, so that the installation point on the module to be installed is aligned with the target installation position point;

[0031] The installation module controls the robotic arm on the auxiliary frame to retrieve the end effector and installation materials corresponding to the module to be installed from within the auxiliary frame, so that the robotic arm can install the module to be installed.

[0032] The present invention also provides an automated installation system for building modules based on robotic arm collaboration, comprising:

[0033] An auxiliary frame is provided, on which at least one robotic arm is fixed. The auxiliary frame includes a receiving space containing various end effectors and installation materials. The end effectors are used to be installed on the robotic arm so that the robotic arm can perform corresponding installation actions. The auxiliary frame is connected to the module to be installed and is located above the module.

[0034] The hoisting equipment is connected to the auxiliary frame via hoisting ropes to control the movement of the auxiliary frame and the module to be installed together.

[0035] Controller, the controller is used for:

[0036] The hoisting equipment is controlled to move the module to be installed to an adjustment position, and the distance between the adjustment position and the target installation position of the module to be installed is within a preset range;

[0037] A target image is acquired, and an alignment result between the module to be installed and the installation position is determined based on the target image. The alignment result reflects the difference between the installation point of the module to be installed and the target installation position point. The target image is obtained by imaging the module to be installed and the target installation position point.

[0038] Based on the alignment result, the hoisting equipment is controlled to move the module to be installed to the target position so that the installation point on the module to be installed is aligned with the target installation position point;

[0039] The robotic arm on the auxiliary frame is controlled to retrieve the end effector and installation materials corresponding to the module to be installed from within the auxiliary frame, so that the robotic arm can install the module to be installed.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automated installation method of building modules based on robotic arm collaboration as described above.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automated installation method for building modules based on robotic arm collaboration as described above.

[0042] The present invention provides an automated installation method for building modules based on robotic arm collaboration. By placing the robotic arm, the installation materials corresponding to the module to be installed, and the end effector of the robotic arm corresponding to the module to be installed in an auxiliary frame, the auxiliary frame and the module to be installed are connected and hoisted together to the installation location. Images are acquired by an image acquisition device set on the auxiliary frame, and the installation point is aligned by image analysis. After the module to be installed reaches the target installation position, the robotic arm on the auxiliary frame retrieves the end effector and installation materials corresponding to the module to be installed from the auxiliary frame for installation. This realizes the automatic installation of the module to be installed by the robotic arm without the need for a large amount of manual support. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the automated installation method for building modules based on robotic arm collaboration provided by the present invention.

[0045] Figure 2 This is a schematic diagram of the auxiliary frame in the automated installation method for building modules based on robotic arm collaboration provided by the present invention;

[0046] Figure 3 This is a schematic diagram of the automated installation system for building modules based on robotic arm collaboration provided by the present invention;

[0047] Figure 4 This is a structural schematic diagram of the automated installation device for building modules based on robotic arm collaboration provided by the present invention;

[0048] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] The following is combined Figures 1-2 The present invention describes the automated installation method for building modules based on robotic arm collaboration, such as... Figure 1 As shown, the method includes the following steps:

[0051] S110. Control the hoisting equipment to move the module to be installed to the adjustment position. The distance between the adjustment position and the target installation point of the module to be installed is within a preset range. The module to be installed is connected to the auxiliary frame and the auxiliary frame is above the module to be installed. At least one robotic arm is fixed on the auxiliary frame.

[0052] The module to be installed is a building module that needs to be installed. It needs to be installed onto an existing building foundation (e.g., another building module that has already been installed). During installation, the mounting points on the module to be installed need to be aligned with the target mounting points on the existing building foundation before the module is secured. In existing technology, after the module to be installed is hoisted above the building foundation, manual adjustment and alignment are required before workers can install it, which is labor-intensive. The method provided by this invention, however, includes an auxiliary frame, such as... Figure 2 As shown, a robotic arm is mounted on the auxiliary frame. Figure 2(The space for accommodating the end effector and installation materials in the auxiliary frame is omitted.) The auxiliary frame and the module to be installed are connected, for example, by multiple hoisting ropes, so that the auxiliary frame and the module to be installed can be moved together. The hoisting equipment first moves the module to be installed and the auxiliary frame to the vicinity of the target installation position of the module to be installed. Specifically, the distance between the adjustment position and the target installation position can be measured by the distance between the adjustment position and the building foundation of the module to be installed. That is, controlling the hoisting equipment to move the module to the adjustment position means controlling the hoisting equipment to move the module to a position where the distance from the building foundation of the module to be installed is not less than a distance threshold. The distance threshold can be determined based on the installation accuracy requirements of the module to be installed. When the installation accuracy of the module to be installed is relatively high, the distance threshold can be set larger, providing sufficient fine-tuning space. When the installation accuracy of the module to be installed is relatively low, the distance threshold can be set smaller, accelerating the installation efficiency.

[0053] S120. Acquire a target image, and determine the alignment result between the mounting point on the module to be installed and the target mounting location point based on the target image. The alignment result reflects the difference between the mounting point on the module to be installed and the target mounting location point. The target image is obtained by imaging the module to be installed and the target mounting location point.

[0054] S130. Based on the alignment result, control the hoisting equipment to move the module to be installed to the target position so that the installation point on the module to be installed is aligned with the target installation position point.

[0055] After the module to be installed reaches the vicinity of the building foundation, the image acquisition device acquires images of the module to be installed and the building foundation to obtain the target image. The image acquisition device can be fixed to the auxiliary frame or to a hoisting device. Determining the alignment result between the installation point on the module to be installed and the target installation location point based on the target image includes:

[0056] The target image is input into a trained image processing model to obtain the installation point detection result output by the image processing model. The installation point detection result includes the installation point and the target installation location point identified from the target image.

[0057] The alignment result is generated based on the installation point detection result, and the alignment result includes the movement direction of the module to be installed.

[0058] Based on the installation point detection results output by the image processing model, the relative position between the installation point on the module to be installed and the target installation location can be determined. Since the target image is planar, the installation point detection results obtained from detecting the installation point on the target image actually only include the relative position between the installation point and the target installation location on the horizontal plane. For example, the installation point may be in front of, behind, to the left of, or to the right of the target installation location. Based on the installation point detection results, the alignment result including the movement direction of the module to be installed can be obtained. For example, the alignment result includes moving the module to be installed forward in the horizontal direction. In other words, the movement direction actually only includes directions on the horizontal plane.

[0059] The step of controlling the hoisting equipment to move the module to be installed to the target position based on the alignment result includes:

[0060] The hoisting equipment is controlled to move the module to be installed in the direction of movement according to the alignment result with a first moving step.

[0061] The hoisting equipment is controlled to move the module to be installed vertically downwards by a preset second moving step.

[0062] Update the target image, and update the installation point detection result based on the updated target image, and update the movement direction and the first movement step size based on the updated installation point detection result;

[0063] The step of controlling the hoisting equipment to move the module to be installed in the direction of movement in the alignment result with a first step size is re-executed until the installation point in the installation point detection result coincides with the target installation position point.

[0064] Control the hoisting equipment to move the module to be installed vertically so that the installation module reaches the target position.

[0065] Determining specific dimensions based on a planar image is a complex process and the results are not necessarily accurate. In the method provided by this invention, the alignment result only includes the direction of movement and not the distance of movement. Instead, a step-by-step adjustment is adopted. Specifically, after obtaining the direction of movement, the hoisting equipment is first controlled to move the module to be installed in accordance with the direction of movement by a preset first step size. In addition, in order to improve the installation efficiency, the hoisting equipment is also controlled to move the module to be installed vertically downward by a preset second step size, so that the module to be installed is closer to the building foundation in the vertical direction.

[0066] After each movement of the module to be installed in the stated direction, the target image is updated. That is, the target image is reacquired, the movement direction is redefined based on the target image, and then the module to be installed is moved according to the new movement direction. It is worth noting that after reacquiring the target image, a step to update the first movement step size is performed. Specifically, since the alignment result only contains direction and not distance, it is difficult to move the module to be installed accurately. However, after one movement, a reference is provided, and the first movement step size can be updated to achieve more accurate movement. The initial value of the first movement step size can be preset. Updating the first movement step size based on the updated installation point detection result includes:

[0067] Obtain the first coordinate position of the installation point in the updated target image from the updated installation point detection result, and obtain the second coordinate position of the installation point in the target image before the update from the original installation point detection result;

[0068] Obtain the first pixel distance between the first coordinate position and the second coordinate position;

[0069] Obtain the ratio between the first movement step size and the first pixel distance before the update;

[0070] Obtain the third coordinate position of the target installation location point in the updated target image from the updated installation point detection results;

[0071] Obtain the second pixel distance between the third coordinate position and the first coordinate position, and multiply the second pixel distance by the ratio to obtain the median distance;

[0072] The updated first movement step size is obtained by dividing the intermediate distance by a preset multiple.

[0073] Specifically, when the intermediate distance is less than the preset minimum value, the first moving step size is directly updated to the intermediate distance.

[0074] After multiple movements, once the installation point in the acquired target image coincides with the target installation location point, the module to be installed is moved vertically downwards and fixed on the building foundation.

[0075] Before controlling the hoisting equipment to move the module to be installed in the direction of movement according to the alignment result by a first moving step, the method provided by the present invention further includes:

[0076] Obtain the horizontal information sent by the horizontal sensor in the robotic arm;

[0077] Control the hoisting equipment and the horizontal information to rotate the module to be installed.

[0078] Specifically, the robotic arm is fixed to the auxiliary frame. Therefore, the level sensor on the robotic arm can reflect whether the auxiliary frame is level. The auxiliary frame and the module to be installed are connected by multiple hoisting ropes. The multiple hoisting ropes connecting the auxiliary frame and the module to be installed are of the same length. Thus, the level of the auxiliary frame is consistent with the level of the module to be installed. Based on the level information sent by the level sensor in the robotic arm, the hoisting equipment can loosen or tighten the connecting ropes to the auxiliary frame, thereby rotating the auxiliary frame and driving the module to be installed to rotate.

[0079] Please refer to it again. Figure 1 The method provided by this invention further includes the following steps:

[0080] S140. Control the robotic arm on the auxiliary frame to obtain the end effector and installation materials corresponding to the module to be installed from the auxiliary frame so that the robotic arm can install the module to be installed.

[0081] The process of controlling the robotic arm on the auxiliary frame to retrieve the end effector corresponding to the module to be installed and the installation materials from within the auxiliary frame so that the robotic arm can install the module to be installed includes:

[0082] Obtain the building module category of the module to be installed, and determine the end effector information and installation material information corresponding to the module to be installed based on the building module category;

[0083] Based on the end effector information, the robotic arm is controlled to retrieve the end effector corresponding to the module to be installed from the auxiliary frame and connect it to the robotic arm. Based on the installation material information, the robotic arm is controlled to retrieve the installation material from the auxiliary frame.

[0084] Acquire an installation image, and acquire the installation point identification result in the installation image. The installation point identification result includes the position of the installation point of the module to be installed in the installation image. The installation image is an image obtained by taking a picture of the module to be installed after it has been moved to the target position.

[0085] Based on the installation point identification result, the robotic arm is controlled to install the module to be installed.

[0086] The auxiliary frame includes a receiving space that holds various end effectors and installation materials. Depending on the building module, the operation for installing the module differs, such as filling or bolting. Different end effectors are required for different operations performed by the robotic arm. Based on the building module category corresponding to the module to be installed, the end effector information and installation material information for that module can be determined. The end effector information indicates what end effector is needed for installing the module, and the installation material information indicates what installation materials are needed. Various end effectors and installation materials are placed within the auxiliary frame. After the module to be installed is aligned and placed on the building foundation, the robotic arm fixed to the auxiliary frame retrieves the corresponding end effector from the receiving space, uses the end effector to grasp the installation materials, and performs the installation operation. For example, when the module needs to be installed using bolts, the robotic arm retrieves the end effector (screwdriver head, drill bit, etc.) and the corresponding installation material (bolt) from the receiving space, and performs installation based on the installation point identified by the module's installation point recognition results.

[0087] The end effector within the auxiliary frame includes at least one of a filling actuator, a cutting actuator, a cleaning actuator, and a fastening actuator. The filling actuator is used to inject building materials into the connection between the module to be installed and the building foundation. The cutting actuator is used to cut excess building materials from the module to be installed. The cleaning actuator is used to clean the connection between the module to be installed and the building foundation. The fastening actuator is used to fasten the fasteners in the module to be installed.

[0088] The installation operation of the module to be installed may include a filling operation, i.e., pouring building materials such as cement into a portion of the space within the module; it may also include a cutting operation, cutting off excess reinforcing bars and other building materials at the connection between the module and the building foundation; the installation operation may also include a cleaning operation, which is required after cutting the reinforcing bars or after filling. For any installation operation that may occur within the building module, the corresponding end effector can be placed within the auxiliary frame. Taking a filling actuator as an example, the filling actuator can be a filling nozzle, connected to a filling source via a filling channel, the filling source storing a large amount of filling material (e.g., cement). The robotic arm can control the opening and closing of the filling actuator's outlet to achieve the filling. Cutting actuators, cleaning actuators, etc., can also be designed accordingly based on the actions they are to perform. Since the actuators corresponding to various operations are placed within the auxiliary frame, the robotic arm can be controlled to connect with the corresponding end effector according to the different types of modules to be installed. Specifically, a corresponding placement position can be set for each type of end effector, and the placement position coordinates can be sent to the robotic arm. The robotic arm can then automatically connect with the end effector and perform corresponding operations after connection, thereby realizing the automated on-site hoisting and installation of building modules, which is impossible in the prior art.

[0089] In summary, the automated installation method for building modules based on robotic arm collaboration provided by this invention involves placing the robotic arm, the installation materials corresponding to the module to be installed, and the end effector of the robotic arm corresponding to the module to be installed within an auxiliary frame. After connecting the auxiliary frame and the module to be installed, they are hoisted together to the installation location. Images are acquired by an image acquisition device set on the auxiliary frame, and the installation point is aligned through image analysis. After the module to be installed reaches the target installation position, the robotic arm on the auxiliary frame retrieves the end effector and installation materials corresponding to the module to be installed from the auxiliary frame for installation. This achieves automated installation of the module to be installed using a robotic arm, without requiring extensive manual support.

[0090] The automated installation system for building modules based on robotic arm collaboration provided by this invention is described below, such as... Figure 3As shown, the automated installation system for building modules based on robotic arm collaboration provided by the present invention is used to install a module 31 to be installed. The system includes an auxiliary frame 32, a hoisting device 33, and a controller. At least one robotic arm is fixed on the auxiliary frame 32. The auxiliary frame 32 includes a receiving space, which contains various end effectors and installation materials. The end effectors are used to be installed on the robotic arm so that the robotic arm can complete the corresponding installation action. The auxiliary frame 32 is connected to the module 31 to be installed and is located above the module 31. The hoisting device 33 is connected to the auxiliary frame 32 through hoisting ropes to control the movement of the auxiliary frame 32 and the module 31 to be installed together. The controller is configured to: control the hoisting equipment 33 to move the module 31 to be installed to an adjustment position, wherein the distance between the adjustment position and the target installation position of the module 31 is within a preset range; acquire a target image, and determine the alignment result between the module to be installed and the installation position based on the target image, wherein the alignment result reflects the difference between the installation point of the module 31 to be installed and the target installation position, and the target image is obtained by imaging the module 31 to be installed and the target installation position; control the hoisting equipment 33 to move the module 31 to be installed to the target position based on the alignment result, so that the installation point on the module 31 to be installed is aligned with the target installation position; and control the robotic arm on the auxiliary frame 32 to retrieve the end effector and installation materials corresponding to the module 31 to be installed from the auxiliary frame 32 so that the robotic arm installs the module 31 to be installed. The hoisting equipment 31 and the robotic arm are both communicatively connected to the controller, which enables the controller to send information and instructions to the hoisting equipment 31 and the robotic arm to realize the installation process of the module to be installed 31. The specific installation process of the module to be installed 31 can be referred to the automated installation method of building modules based on robotic arm collaboration described above.

[0091] The automated installation device for building modules based on robotic arm collaboration provided by the present invention is described below. The automated installation device for building modules based on robotic arm collaboration described below can be referred to in correspondence with the automated installation method for building modules based on robotic arm collaboration described above.

[0092] like Figure 4 As shown, the automated installation device for building modules based on robotic arm collaboration provided by the present invention includes:

[0093] A coarse movement module 410 is used to control the hoisting equipment to move the module to be installed to an adjustment position. The distance between the adjustment position and the target installation position of the module to be installed is not less than a distance threshold. The module to be installed is connected to an auxiliary frame and the auxiliary frame is above the module to be installed. At least one robotic arm is fixed on the auxiliary frame.

[0094] Image processing module 420 is used to acquire a target image and determine the alignment result between the mounting point on the module to be installed and the target mounting position point based on the target image. The alignment result reflects the difference between the mounting point on the module to be installed and the target mounting position point. The target image is obtained by imaging the module to be installed and the target mounting position point.

[0095] Fine movement module 430, the fine movement module 430 is used to control the hoisting equipment to move the module to be installed to the target position based on the alignment result, so that the installation point on the module to be installed is aligned with the target installation position point;

[0096] Installation module 440 is used to control the robotic arm on the auxiliary frame to obtain the end effector and installation materials corresponding to the module to be installed from the auxiliary frame so that the robotic arm can install the module to be installed.

[0097] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an automated installation method for building modules based on robotic arm collaboration. The method includes: controlling a hoisting device to move the module to be installed to an adjustment position, wherein the distance between the adjustment position and the target installation position of the module to be installed is not less than a distance threshold, the module to be installed is connected to an auxiliary frame and the auxiliary frame is above the module to be installed, and at least one robotic arm is fixed on the auxiliary frame.

[0098] A target image is acquired, and the alignment result between the mounting point on the module to be installed and the target mounting position point is determined based on the target image. The alignment result reflects the difference between the mounting point on the module to be installed and the target mounting position point. The target image is obtained by imaging the module to be installed and the target mounting position point.

[0099] Based on the alignment result, the hoisting equipment is controlled to move the module to be installed to the target position so that the installation point on the module to be installed is aligned with the target installation position point;

[0100] The robotic arm on the auxiliary frame is controlled to retrieve the end effector and installation materials corresponding to the module to be installed from within the auxiliary frame, so that the robotic arm can install the module to be installed.

[0101] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automated installation method for building modules based on robotic arm collaboration provided by the above methods. The method includes: controlling a hoisting device to move a module to be installed to an adjustment position, wherein the distance between the adjustment position and the target installation position of the module to be installed is not less than a distance threshold, the module to be installed is connected to an auxiliary frame and the auxiliary frame is above the module to be installed, and at least one robotic arm is fixed on the auxiliary frame.

[0103] A target image is acquired, and the alignment result between the mounting point on the module to be installed and the target mounting position point is determined based on the target image. The alignment result reflects the difference between the mounting point on the module to be installed and the target mounting position point. The target image is obtained by imaging the module to be installed and the target mounting position point.

[0104] Based on the alignment result, the hoisting equipment is controlled to move the module to be installed to the target position so that the installation point on the module to be installed is aligned with the target installation position point;

[0105] The robotic arm on the auxiliary frame is controlled to retrieve the end effector and installation materials corresponding to the module to be installed from within the auxiliary frame, so that the robotic arm can install the module to be installed.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automated installation of building modules based on cooperation of robotic arms, characterized by, The method comprises: controlling a hoisting device to move a to-be-installed module to an adjusting position, a distance between the adjusting position and a target installation position point of the to-be-installed module is not less than a distance threshold, the to-be-installed module is connected with an auxiliary frame, and the auxiliary frame is above the to-be-installed module, and at least one mechanical arm is fixed on the auxiliary frame; obtaining a target image, determining an alignment result of an installation point on the to-be-installed module and the target installation position point based on the target image, the alignment result reflecting a difference between the installation point on the to-be-installed module and the target installation position point, and the target image being obtained by imaging the to-be-installed module and the target installation position point; controlling the hoisting device to move the to-be-installed module to a target position based on the alignment result, so that the installation point on the to-be-installed module is aligned with the target installation position point; controlling the mechanical arm on the auxiliary frame to obtain an end effector corresponding to the to-be-installed module and installation materials from the auxiliary frame, so that the mechanical arm installs the to-be-installed module; the method comprises: inputting the target image into a trained image processing model, obtaining an installation point detection result output by the image processing model, and the installation point detection result including the installation point and the target installation position point identified from the target image; generating the alignment result based on the installation point detection result, and the alignment result including a moving direction of the to-be-installed module; the method comprises: controlling the hoisting device to move the to-be-installed module in the moving direction in the alignment result at a first moving step; controlling the hoisting device to move the to-be-installed module vertically downward at a preset second moving step; updating the target image, updating the installation point detection result based on the updated target image, updating the moving direction and the first moving step based on the updated installation point detection result; re-executing the step of controlling the hoisting device to move the to-be-installed module in the moving direction in the alignment result at the first moving step until the installation point in the installation point detection result coincides with the target installation position point; controlling the hoisting device to move the to-be-installed module vertically to make the to-be-installed module reach the target position; the method comprises: obtaining a first coordinate position of the installation point in the updated target image in the updated installation point detection result, and obtaining a second coordinate position of the installation point in the previous target image in the previous installation point detection result; obtaining a first pixel distance between the first coordinate position and the second coordinate position; obtaining a ratio between the first moving step before the update and the first pixel distance; obtaining a third coordinate position of the target installation position point in the updated target image in the updated installation point detection result; obtaining a second pixel distance between the third coordinate position and the first coordinate position, and multiplying the second pixel distance by the ratio to obtain an intermediate distance; dividing the intermediate distance by a preset multiple to obtain an updated first movement step.

2. The robotic arm collaboration based automated installation method of building modules as claimed in claim 1, wherein, Before the control of the hoisting device to move the to-be-installed module by the first movement step according to the movement direction in the alignment result, the method further comprises: obtaining horizontal information sent by a horizontal sensor in the mechanical arm; controlling the hoisting device to rotate the to-be-installed module based on the horizontal information.

3. The robotic arm collaboration based building module automated installation method of claim 1, wherein, The end effector in the auxiliary frame comprises at least one of a filling effector, a cutting effector, a cleaning effector, and a fastening effector, wherein the filling effector is used to fill building materials to a connection between the to-be-installed module and a building foundation, the cutting effector is used to cut excess building materials of the to-be-installed module, the cleaning effector is used to clean the connection between the to-be-installed module and the building foundation, and the fastening effector is used to perform a fastening operation on a fastener in the to-be-installed module.

4. The robotic arm collaboration based automated installation method of building modules as claimed in claim 1, wherein, The control of the mechanical arm on the auxiliary frame to obtain the corresponding end effector and installation materials of the to-be-installed module from the auxiliary frame so that the mechanical arm installs the to-be-installed module comprises: obtaining a building module category of the to-be-installed module, and determining end effector information and installation material information corresponding to the to-be-installed module based on the building module category; controlling the mechanical arm to obtain the corresponding end effector connected behind the mechanical arm from the auxiliary frame based on the end effector information, and controlling the mechanical arm to obtain installation materials from the auxiliary frame based on the installation material information; obtaining an installation image, and obtaining an installation point recognition result in the installation image, the installation point recognition result comprising a position of an installation point of the to-be-installed module in the installation image, the installation image being an image obtained by photographing the to-be-installed module after the to-be-installed module moves to the target position; controlling the mechanical arm to install the to-be-installed module based on the installation point recognition result.

5. A construction module automated installation device based on mechanical arm cooperation, characterized in that, The device comprises: a coarse movement module configured to control a hoisting device to move a to-be-installed module to an adjustment position, a distance between the adjustment position and a target installation position point of the to-be-installed module being not less than a distance threshold, the to-be-installed module being connected with an auxiliary frame and the auxiliary frame being above the to-be-installed module, and at least one mechanical arm being fixed on the auxiliary frame; an image processing module configured to obtain a target image, determine an alignment result of an installation point on the to-be-installed module and the target installation position point based on the target image, the alignment result reflecting a difference between the installation point on the to-be-installed module and the target installation position point, and the target image being obtained by imaging the to-be-installed module and the target installation position point. a fine movement module configured to control the hoisting device to move the to-be-installed module to a target position based on the alignment result, so that a mounting point on the to-be-installed module is aligned with the target mounting position point; an installation module configured to control a mechanical arm on the auxiliary frame to obtain an end effector corresponding to the to-be-installed module and installation materials from the auxiliary frame, so that the mechanical arm installs the to-be-installed module; the alignment result of the mounting point on the to-be-installed module and the target mounting position point based on the target image includes: inputting the target image into a trained image processing model, and obtaining a mounting point detection result output by the image processing model, wherein the mounting point detection result includes the mounting point and the target mounting position point identified from the target image; generating the alignment result based on the mounting point detection result, wherein the alignment result includes a movement direction of the to-be-installed module; the control of the hoisting device to move the to-be-installed module to a target position based on the alignment result includes: controlling the hoisting device to move the to-be-installed module in the movement direction in the alignment result by a first movement step; controlling the hoisting device to move the to-be-installed module vertically downward by a preset second movement step; updating the target image, and updating the mounting point detection result based on the updated target image, updating the movement direction and the first movement step based on the updated mounting point detection result; re-executing the step of controlling the hoisting device to move the to-be-installed module in the movement direction in the alignment result by the first movement step until the mounting point in the mounting point detection result coincides with the target mounting position point; controlling the hoisting device to move the to-be-installed module vertically so that the installation module reaches the target position; the updating of the first movement step based on the updated mounting point detection result includes: obtaining a first coordinate position of the mounting point in the updated target image in the updated mounting point detection result, and obtaining a second coordinate position of the mounting point in the target image before updating in the mounting point detection result before updating; obtaining a first pixel distance between the first coordinate position and the second coordinate position; obtaining a ratio between the first movement step before updating and the first pixel distance; obtaining a third coordinate position of the target mounting position point in the updated target image in the updated mounting point detection result; obtaining a second pixel distance between the third coordinate position and the first coordinate position, and multiplying the second pixel distance by the ratio to obtain an intermediate distance; dividing the intermediate distance by a preset multiple to obtain the first movement step after updating.

6. A robotic arm collaboration based building module automated installation system for installing a module to be installed; characterized in that, the system includes: An auxiliary frame on which at least one mechanical arm is fixed, the auxiliary frame comprising a containing space in which a plurality of end effectors and installation materials are contained, the end effectors being used to be installed on the mechanical arm to enable the mechanical arm to complete corresponding installation actions, the auxiliary frame being connected with the to-be-installed module and being above the installation module; A hoisting device connecting the auxiliary frame through a hoisting rope to control the auxiliary frame and the to-be-installed module to move together; A controller configured to: control the hoisting device to move the to-be-installed module to an adjusting position, the distance between the adjusting position and a target installation position point of the to-be-installed module being within a preset range; obtain a target image, determine an alignment result of the to-be-installed module and the installation position based on the target image, the alignment result reflecting the difference between an installation point on the to-be-installed module and the target installation position point, the target image being obtained by imaging the to-be-installed module and the target installation position point; control the hoisting device to move the to-be-installed module to a target position based on the alignment result, so that the installation point on the to-be-installed module is aligned with the target installation position point; control the mechanical arm on the auxiliary frame to obtain the corresponding end effector and installation material of the to-be-installed module from the auxiliary frame to enable the mechanical arm to install the to-be-installed module; the determination of the alignment result of the installation point on the to-be-installed module and the target installation position point based on the target image comprises: inputting the target image into a trained image processing model, obtaining an installation point detection result output by the image processing model, the installation point detection result including the installation point and the target installation position point identified from the target image; generating the alignment result based on the installation point detection result, the alignment result including a moving direction of the to-be-installed module; the control of the hoisting device to move the to-be-installed module to a target position based on the alignment result comprises: controlling the hoisting device to move the to-be-installed module in the moving direction in the alignment result at a first moving step; controlling the hoisting device to move the to-be-installed module vertically downward at a preset second moving step; updating the target image, and updating the installation point detection result based on the updated target image, updating the moving direction and the first moving step based on the updated installation point detection result; re-executing the step of controlling the hoisting device to move the to-be-installed module in the moving direction in the alignment result at a first moving step until the installation point in the installation point detection result coincides with the target installation position point; controlling the hoisting device to move the to-be-installed module vertically to enable the installation module to reach the target position; the updating of the first moving step based on the updated installation point detection result comprises: obtain a first coordinate position of the installation point in the updated target image in the updated installation point detection result, and obtain a second coordinate position of the installation point in the target image before the update in the installation point detection result before the update; obtain a first pixel distance between the first coordinate position and the second coordinate position; obtain a ratio between the first moving step length before the update and the first pixel distance; obtain a third coordinate position of the target installation position point in the updated target image in the updated installation point detection result; obtain a second pixel distance between the third coordinate position and the first coordinate position, and multiply the second pixel distance by the ratio to obtain an intermediate distance; divide the intermediate distance by a preset multiple to obtain the first moving step length after the update.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the computer program to implement the method for automatic installation of building modules based on cooperation of mechanical arms according to any one of claims 1 to 4.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method for automatic installation of building modules based on cooperation of mechanical arms according to any one of claims 1 to 4.

Images