Outrigger assembly method, engineering machine, storage medium and processor

Abstract

Embodiments of the present application provide a method for assembling outrigger, a construction machine, a processor and a storage medium. The method comprises: acquiring an outrigger image of an outrigger to be assembled by an image acquisition device, the image acquisition device being connected to the end of a multi-axis robot; for each outrigger image, determining a first spatial position of the image acquisition device, a distance between the image acquisition device and the outrigger to be assembled, and a shooting angle of the image acquisition device; for each outrigger image, determining a second spatial position of each detection point according to the first spatial position, the distance and the shooting angle; determining a deviation value between the first outrigger and the second outrigger according to the second spatial position of all detection points of the outrigger to be assembled; and controlling the center points of the first outrigger and the second outrigger to align in the length direction according to the deviation value to perform the assembly operation. The image acquisition device can flexibly acquire outrigger images in multiple directions, calibrate the outrigger, and accurately perform the assembly operation.

Description

Technical Field

[0001] This application relates to the field of intelligent assembly technology, specifically to a leg assembly method, engineering machinery, storage medium, and processor. Background Technology

[0002] Before assembly, the primary and secondary movable outriggers of cranes are typically not dimensionally inspected, or only critical dimensions are inspected using fixed-position cameras. Because the measuring cameras are fixed, their measurable positions and ranges are limited, making them unsuitable for mixed-line production of workpieces with significant differences in shape and size. Furthermore, fixed-position cameras are prone to interference with tooling and workpiece hoisting processes, and also have blind spots, making them unsuitable for assembling outriggers with numerous product models and significant dimensional variations. Summary of the Invention

[0003] The purpose of this application is to provide a leg assembly method, engineering machinery, storage medium, and processor.

[0004] To achieve the above objectives, the first aspect of this application provides a method for assembling an outrigger, comprising:

[0005] The outrigger image to be assembled is acquired by an image acquisition device. The outrigger to be assembled includes a first outrigger and a second outrigger. The image acquisition device is connected to the end effector of the multi-axis robotic arm.

[0006] For each outrigger image, determine the first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device;

[0007] For each leg image, the second spatial position of each detection point is determined based on the first spatial position, the distance between each detection point, and the shooting angle.

[0008] The deviation value between the first and second legs is determined based on the second spatial position of all the detection points of the legs to be assembled.

[0009] The center points of the first leg and the second leg are aligned along the length direction based on the deviation value in order to carry out the assembly operation.

[0010] In embodiments of this application, aligning the center points of the first leg and the second leg in the length direction based on the deviation value for assembly operations includes: determining the motion offset of the servo mechanism based on the deviation value; controlling the servo mechanism to move based on the motion offset to keep the second leg stationary and control the center point of the first leg to move toward the center point of the second leg; and determining that the center points of the first leg and the second leg are aligned in the length direction if the deviation value between the center points of the first leg and the second leg is less than or equal to a preset deviation threshold.

[0011] In embodiments of this application, the method further includes: acquiring the model number of the support leg to be assembled before acquiring the support leg image of the support leg to be assembled through the image acquisition device; determining the calibration program corresponding to the support leg to be assembled based on the support leg model; determining the movement trajectory and shooting angle of the image acquisition device based on the calibration program; controlling the image acquisition device to move according to the movement trajectory and acquiring the support leg image of the support leg to be assembled at the shooting angle.

[0012] In the embodiments of this application, for each outrigger image, determining the second spatial position of each detection point based on the first spatial position, the distance between them, and the second spatial position of each detection point includes: determining the image position of each detection point in each outrigger image and the device parameters of the image acquisition device; for each outrigger image, determining the homogeneous coordinates of each detection point based on the image position of each detection point and the device parameters, where homogeneous coordinates refer to spatial coordinates in the camera coordinate system with the origin of the first spatial position; and for each outrigger image, converting the homogeneous coordinates of each detection point into the second spatial position based on the distance between them and the shooting angle.

[0013] In the embodiments of this application, for each leg image, converting the homogeneous coordinates of each detection point into a second spatial position based on the distance between them and the shooting angle includes: for each leg image, determining the translation vector and rotation matrix of the image acquisition device based on the distance between them and the shooting angle; and converting the homogeneous coordinates of each detection point into a second spatial position based on the translation vector and the rotation matrix.

[0014] In the embodiments of this application, the device parameters include the focal length and optical center position of the image acquisition device. For each leg image, determining the homogeneous coordinates of each detection point based on the image position and device parameters includes: for each leg image, converting the image position of each detection point into homogeneous coordinates of each detection point based on the focal length and optical center position, wherein the homogeneous coordinates of each detection point are calculated according to the following formula (1):

[0015]

[0016] Among them, f x This refers to the focal length of the image acquisition device in the x-direction, f. y This refers to the focal length of the image acquisition device in the y-direction, c. x and c y [uv] refers to the position of the optical center of the image acquisition device, [uv] refers to the image position of each detection point in each leg image, [x] refers to the position of the optical center of the image acquisition device. c y c z c [] refers to the homogeneous coordinates of each detection point in each leg image. Homogeneous coordinates are spatial coordinates in the camera coordinate system with the origin of the first spatial position.

[0017] A second aspect of this application provides a processor configured to perform the above-described leg assembly method.

[0018] A third aspect of this application provides an engineering machine, comprising:

[0019] An image acquisition device is connected to the end effector of a multi-axis robotic arm. The image acquisition device is used to acquire images of the outriggers to be assembled.

[0020] Multi-axis robotic arms are used to control the movement trajectory and shooting angle of image acquisition equipment;

[0021] And a processor configured to perform the above-described outrigger assembly method.

[0022] A fourth aspect of this application provides a machine-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the aforementioned leg assembly method.

[0023] Using the aforementioned outrigger assembly method, engineering machinery, storage medium, and processor, an image acquisition device acquires images of the outriggers to be assembled. The outriggers include a first outrigger and a second outrigger. The image acquisition device is connected to the end effector of a multi-axis robotic arm. For each outrigger image, the first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device are determined. For each outrigger image, the second spatial position of each detection point is determined based on the first spatial position, distance, and shooting angle. The deviation value between the first and second outriggers is determined based on the second spatial positions of all detection points on the outrigger to be assembled. The center points of the first and second outriggers are aligned along their length direction based on the deviation value to facilitate assembly. The image acquisition device can flexibly acquire outrigger images from multiple directions, calibrating the outriggers to be assembled for accurate assembly. Interference with tooling fixtures is reduced during the imaging process. Measurements can be performed on certain corner areas and internal cavities of the outriggers. It is applicable to different outrigger models and improves assembly accuracy.

[0024] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings:

[0026] Figure 1The schematic diagram illustrates a flow chart of a leg assembly method according to an embodiment of this application;

[0027] Figure 2 A schematic diagram of an industrial robot according to an embodiment of this application is shown;

[0028] Figure 3 A schematic diagram of the structure of an engineering machine according to an embodiment of this application is shown.

[0029] Figure 4 The diagram illustrates the internal structure of a computer device according to an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0031] The figure schematically illustrates a flow diagram of a leg assembly method according to an embodiment of this application. Figure 1 As shown, in one embodiment of this application, a leg assembly method is provided, including the following steps:

[0032] S102, acquire the image of the support leg to be assembled through the image acquisition device. The support leg to be assembled includes a first support leg and a second support leg. The image acquisition device is connected to the end of the multi-axis robotic arm.

[0033] S104, for each outrigger image, determine the first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device;

[0034] S106, for each leg image, determine the second spatial position of each detection point based on the first spatial position, the distance between them, and the shooting angle;

[0035] S108, determine the deviation value between the first leg and the second leg based on the second spatial position of all detection points of the outrigger to be assembled;

[0036] S110, according to the deviation value, controls the center point of the first leg and the center point of the second leg to be aligned in the length direction for assembly operation.

[0037] like Figure 2 As shown, Figure 2A schematic diagram of an industrial robot according to an embodiment of this application is shown. The industrial robot has a multi-axis robotic arm that can rotate in various directions. An image acquisition device is mounted at the end of the multi-axis robotic arm to acquire images from multiple angles. A processor can acquire images of the outriggers to be assembled through the image acquisition device. The outrigger to be assembled refers to an outrigger assembly that has not yet been fully assembled, including a first leg and a second leg. It is understood that the size of one of the first and second legs is larger than the size of the other leg so that the two legs can be fitted together. During the movement of the industrial robot, the image acquisition device can acquire images of the outriggers at different shooting angles, and the outrigger images include the first and second legs. For each outrigger image, the processor can determine a first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device. The first spatial position of the image acquisition device can be determined by the position of the industrial robot. The outrigger to be assembled is fixed on a servo mechanism for performing the assembly operation, and the distance refers to the distance between the spatial position of the image acquisition device and the spatial position of the outrigger to be assembled. The shooting angle of the image acquisition device refers to the angle between the shooting direction of the image acquisition device and the initial position of the image acquisition device. Therefore, the processor can determine the second spatial position of each detection point on the outrigger to be assembled in the outrigger image based on the first spatial position, the distance between them, and the shooting angle. The second spatial position refers to the three-dimensional coordinates of the detection point on the outrigger to be assembled. The processor can determine the deviation value between the first and second outriggers based on the second spatial positions of all the detection points on the outrigger to be assembled. The deviation value refers to the distance between the center points of the first and second outriggers along the cross-section of the outrigger. Then, the processor controls the center points of the first and second outriggers to align along their length direction based on the deviation value for assembly operations.

[0038] In one embodiment, aligning the center points of the first leg and the second leg in the length direction based on the deviation value for assembly includes: determining the motion offset of the servo mechanism based on the deviation value; controlling the servo mechanism to move based on the motion offset to keep the second leg stationary and control the center point of the first leg to move toward the center point of the second leg; and determining that the center points of the first leg and the center points of the second leg are aligned in the length direction if the deviation value between the center points of the first leg and the center points of the second leg is less than or equal to a preset deviation threshold.

[0039] In the leg assembly process, the leg to be assembled is first fixed in the fixture of the servo mechanism. The processor can determine the motion offset of the servo mechanism based on the deviation between the first and second legs. The motion offset refers to the distance the servo mechanism components move up, down, left, and right. The processor can then control the up, down, left, and right movements of the servo mechanism components based on the motion offset. During the movement of the servo mechanism, the leg to be assembled also moves. The processor can then control the component fixing the second leg in the servo mechanism to remain stationary and control the center point of the first leg to move towards the center point of the second leg, gradually aligning the first and second legs along their length. The processor can then determine that the center points of the first and second legs are aligned along their length if the deviation between their center points is less than or equal to a preset deviation threshold. The preset deviation threshold refers to the assembly error value of the leg; within this threshold, the first and second legs can complete the assembly process.

[0040] In one embodiment, the method further includes: acquiring the model number of the support leg to be assembled before acquiring the support leg image of the support leg to be assembled through the image acquisition device; determining the calibration program corresponding to the support leg to be assembled based on the support leg model; determining the movement trajectory and shooting angle of the image acquisition device based on the calibration program; controlling the image acquisition device to move according to the movement trajectory and acquiring the support leg image of the support leg to be assembled at the shooting angle.

[0041] Before acquiring images of the outriggers to be assembled using image acquisition equipment, the processor can obtain the outrigger model number. The outrigger model number refers to the material, specifications, and size of the outrigger. Specifically, the outrigger model number can be determined by scanning the QR code (or barcode) on the outrigger using a barcode scanner at the workstation. Then, the processor can determine the calibration program for the outrigger. The calibration program controls the image acquisition and inspection process for the outrigger. Specifically, a corresponding ground-mounted track axis can be configured in the working environment, and the industrial robot is mounted on the ground-mounted track axis. The processor can then determine the movement trajectory and shooting angle of the image acquisition equipment according to the calibration program, thereby controlling the movement of the industrial robot so that the image acquisition equipment acquires outrigger images while moving. Simultaneously, the shooting angle of the image acquisition equipment can be adjusted using a multi-axis robotic arm according to the shooting angle specified in the calibration program to acquire images of the outrigger to be assembled. In this way, the robot's flexibility can be fully utilized, and the measurement position and range can be effectively extended to the robot's travel range. During the shooting process, interference with tooling fixtures is reduced, and measurements can be taken of certain corner areas and internal cavities of the outriggers.

[0042] In one embodiment, for each outrigger image, determining the second spatial position of each detection point based on the first spatial position, the distance between each detection point, and the second spatial position of each detection point includes: determining the image position of each detection point in each outrigger image and the device parameters of the image acquisition device; for each outrigger image, determining the homogeneous coordinates of each detection point based on the image position and device parameters of each detection point, where homogeneous coordinates refer to spatial coordinates in the camera coordinate system with the origin of the first spatial position; and for each outrigger image, converting the homogeneous coordinates of each detection point into the second spatial position based on the distance between the detection points and the shooting angle.

[0043] In one embodiment, the device parameters include the focal length and optical center position of the image acquisition device. For each leg image, determining the homogeneous coordinates of each detection point based on the image position and device parameters includes: for each leg image, converting the image position of each detection point into homogeneous coordinates of each detection point based on the focal length and optical center position, wherein the homogeneous coordinates of each detection point are calculated according to the following formula (1):

[0044]

[0045] Among them, f x This refers to the focal length of the image acquisition device in the x-direction, f. y This refers to the focal length of the image acquisition device in the y-direction, c. z and c y [uv] refers to the position of the optical center of the image acquisition device, [uv] refers to the image position of each detection point in each leg image, [x] refers to the position of the optical center of the image acquisition device. c y c z c [ ] refers to the homogeneous coordinates of each detection point in each outrigger image. Homogeneous coordinates are spatial coordinates in the camera coordinate system with the origin at the first spatial position. Using the above method, the image coordinates of the detection points in the outrigger image can be converted into three-dimensional coordinates in the camera coordinate system.

[0046] In one embodiment, for each outrigger image, converting the homogeneous coordinates of each detection point into a second spatial position based on the distance between them and the shooting angle includes: for each outrigger image, determining the translation vector and rotation matrix of the image acquisition device based on the distance between them and the shooting angle; and converting the homogeneous coordinates of each detection point into a second spatial position based on the translation vector and the rotation matrix.

[0047] For each leg image, the processor can determine the spatial position of the industrial robot when the image acquisition device captures the image, thus determining the spatial position of the image acquisition device. Then, for each leg image, the processor can determine the relative spatial position between the image acquisition device and the leg to be assembled, thus determining the distance between them. Further, the processor can determine the translation vector of the image acquisition device based on the distance and the rotation matrix of the image acquisition device based on the shooting angle. The translation vector represents how to move the origin of one coordinate system to the origin of another coordinate system, or in other words, the translation vector is the offset between the origins of the first and second coordinate systems. Therefore, moving from a coordinate system with the target center as the origin to another coordinate system with the camera center as the origin, the corresponding translation vector T is the relative spatial position parameter between the target origin and the camera origin. The rotation matrix R includes multiple rotation parameters for the coordinate system rotation. Based on the translation vector and the rotation matrix, the processor can transform the homogeneous coordinates of each detection point in the leg image into a second spatial position. In this way, the 3D coordinates of the detection points in the camera coordinate system can be transformed into spatial coordinates in the world coordinate system.

[0048] Using the aforementioned outrigger assembly method, engineering machinery, storage medium, and processor, an image acquisition device acquires images of the outriggers to be assembled. The outriggers include a first outrigger and a second outrigger. The image acquisition device is connected to the end effector of a multi-axis robotic arm. For each outrigger image, the first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device are determined. For each outrigger image, the second spatial position of each detection point is determined based on the first spatial position, the distance, and the shooting angle. The deviation value between the first and second outriggers is determined based on the second spatial positions of all detection points of the outrigger to be assembled. The center points of the first and second outriggers are aligned along their length direction based on the deviation value to perform the assembly operation. The image acquisition device can flexibly acquire outrigger images from multiple directions, converting the detection points of the outriggers to be assembled in the outrigger images from image coordinates to three-dimensional spatial coordinates, thereby calibrating the outriggers to be assembled for accurate assembly. During the shooting process, interference with tooling fixtures is reduced. Measurements can be made on certain corner areas and inner cavities of the outriggers. This method is applicable to different models of outriggers and improves assembly accuracy.

[0049] Figure 1 This is a flowchart illustrating the outrigger assembly method in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0050] In one embodiment, such as Figure 3 As shown, an engineering machine is provided, including:

[0051] Image acquisition device 310 is connected to the end of multi-axis robotic arm 320. Image acquisition device 310 is used to acquire images of the outrigger to be assembled.

[0052] A multi-axis robotic arm 320 is used to control the movement trajectory and shooting angle of the image acquisition device 310;

[0053] The processor 330 is configured to perform the above-described leg assembly method.

[0054] An image acquisition device 310 is mounted at the end of a multi-axis robotic arm 320 to acquire images of the outrigger to be assembled from multiple angles. The multi-axis robotic arm 320 controls the movement trajectory and shooting angle of the image acquisition device 310. A processor 330 can control the multi-axis robotic arm 320 to move along the movement trajectory and rotate according to the shooting angle. Furthermore, the processor can control the image acquisition device 310 to acquire images of the outrigger to be assembled during the movement of the multi-axis robotic arm 320, according to the shooting angle.

[0055] In one embodiment, it further includes a servo mechanism for controlling the movement of the outrigger to be assembled.

[0056] In the outrigger assembly process, the outrigger to be assembled is first fixed in the fixture of the servo mechanism. The servo mechanism is used to control the movement of the outrigger to be assembled. The processor can determine the motion offset of the servo mechanism based on the deviation between the first and second outriggers. Then, the processor can control the components of the servo mechanism to move up, down, left, and right based on the motion offset. During the movement of the servo mechanism, the outrigger to be assembled also moves accordingly. The processor can then control the component fixing the second outrigger in the servo mechanism to remain stationary and control the center point of the first outrigger to move towards the center point of the second outrigger, so that the first and second outriggers gradually align along their length. Therefore, the processor can determine that the center points of the first and second outriggers are aligned along their length if the deviation between the center points of the outriggers is less than or equal to a preset deviation threshold.

[0057] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and the method for assembling the legs can be implemented by adjusting the kernel parameters.

[0058] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0059] This application provides a storage medium storing a program that, when executed by a processor, implements the above-described leg assembly method.

[0060] This application provides a processor for running a program, wherein the program executes the above-described support leg assembly method during runtime.

[0061] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The database is used to store XXX data. The network interface A02 is used for communication with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a leg assembly method.

[0062] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] This application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps:

[0064] Images of the outriggers to be assembled are acquired using an image acquisition device. The outriggers to be assembled include a first outrigger and a second outrigger. The image acquisition device is connected to the end effector of a multi-axis robotic arm. For each outrigger image, the first spatial position of the image acquisition device, the distance between the image acquisition device and each detection point of the outrigger to be assembled, and the shooting angle of the image acquisition device are determined. For each outrigger image, the second spatial position of each detection point is determined based on the first spatial position, the distance between the detection points, and the shooting angle. The deviation value between the first outrigger and the second outrigger is determined based on the second spatial positions of all detection points of the outrigger to be assembled. The center points of the first outrigger and the second outrigger are aligned in the length direction based on the deviation value to perform the assembly operation.

[0065] In one embodiment, aligning the center points of the first leg and the second leg in the length direction based on the deviation value for assembly includes: determining the motion offset of the servo mechanism based on the deviation value; controlling the servo mechanism to move based on the motion offset to keep the second leg stationary and control the center point of the first leg to move toward the center point of the second leg; and determining that the center points of the first leg and the center points of the second leg are aligned in the length direction if the deviation value between the center points of the first leg and the center points of the second leg is less than or equal to a preset deviation threshold.

[0066] In one embodiment, the method further includes: acquiring the model number of the support leg to be assembled before acquiring the support leg image of the support leg to be assembled through the image acquisition device; determining the calibration program corresponding to the support leg to be assembled based on the support leg model; determining the movement trajectory and shooting angle of the image acquisition device based on the calibration program; controlling the image acquisition device to move according to the movement trajectory and acquiring the support leg image of the support leg to be assembled at the shooting angle.

[0067] In one embodiment, determining the second spatial position of each detection point for each outrigger image based on the first spatial position, the distance between them, and the second spatial position of each detection point includes: determining the image position of each detection point in each outrigger image and the device parameters of the image acquisition device; determining the homogeneous coordinates of each detection point for each outrigger image based on the image position of each detection point and the device parameters, wherein the homogeneous coordinates refer to the spatial coordinates in the camera coordinate system with the origin of the first spatial position; and converting the homogeneous coordinates of each detection point into the second spatial position for each outrigger image based on the distance between them and the shooting angle.

[0068] In one embodiment, for each outrigger image, converting the homogeneous coordinates of each detection point into a second spatial position based on the distance between them and the shooting angle includes: for each outrigger image, determining the translation vector and rotation matrix of the image acquisition device based on the distance between them and the shooting angle; and converting the homogeneous coordinates of each detection point into a second spatial position based on the translation vector and the rotation matrix.

[0069] In one embodiment, the device parameters include the focal length and optical center position of the image acquisition device. For each leg image, determining the homogeneous coordinates of each detection point based on the image position and device parameters includes: for each leg image, converting the image position of each detection point into homogeneous coordinates of each detection point based on the focal length and optical center position, wherein the homogeneous coordinates of each detection point are calculated according to the following formula (1):

[0070]

[0071] Among them, f x This refers to the focal length of the image acquisition device in the x-direction, f. y This refers to the focal length of the image acquisition device in the y-direction, c. x and c y [uv] refers to the position of the optical center of the image acquisition device, [uv] refers to the image position of each detection point in each leg image, [x] refers to the position of the optical center of the image acquisition device. c y c z c [] refers to the homogeneous coordinates of each detection point in each leg image. Homogeneous coordinates are spatial coordinates in the camera coordinate system with the origin of the first spatial position.

[0072] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the following method steps:

[0073] Images of the outriggers to be assembled, including a first leg and a second leg, are acquired using an image acquisition device connected to the end effector of a multi-axis robotic arm. For each outrigger image, a first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device are determined. For each outrigger image, a second spatial position of each detection point is determined based on the first spatial position, the distance between the detection points, and the shooting angle. The deviation value between the first leg and the second leg is determined based on the second spatial positions of all detection points of the outrigger to be assembled. The center points of the first leg and the second leg are aligned along their length direction based on the deviation value to perform the assembly operation.

[0074] In one embodiment, aligning the center points of the first leg and the second leg in the length direction based on the deviation value for assembly includes: determining the motion offset of the servo mechanism based on the deviation value; controlling the servo mechanism to move based on the motion offset to keep the second leg stationary and control the center point of the first leg to move toward the center point of the second leg; and determining that the center points of the first leg and the center points of the second leg are aligned in the length direction if the deviation value between the center points of the first leg and the center points of the second leg is less than or equal to a preset deviation threshold.

[0075] In one embodiment, the method further includes: acquiring the model number of the support leg to be assembled before acquiring the support leg image of the support leg to be assembled through the image acquisition device; determining the calibration program corresponding to the support leg to be assembled based on the support leg model; determining the movement trajectory and shooting angle of the image acquisition device based on the calibration program; controlling the image acquisition device to move according to the movement trajectory and acquiring the support leg image of the support leg to be assembled at the shooting angle.

[0076] In one embodiment, determining the second spatial position of each detection point for each outrigger image based on the first spatial position, the distance between them, and the second spatial position of each detection point includes: determining the image position of each detection point in each outrigger image and the device parameters of the image acquisition device; determining the homogeneous coordinates of each detection point for each outrigger image based on the image position of each detection point and the device parameters, wherein the homogeneous coordinates refer to the spatial coordinates in the camera coordinate system with the origin of the first spatial position; and converting the homogeneous coordinates of each detection point into the second spatial position for each outrigger image based on the distance between them and the shooting angle.

[0077] In one embodiment, for each outrigger image, converting the homogeneous coordinates of each detection point into a second spatial position based on the distance between them and the shooting angle includes: for each outrigger image, determining the translation vector and rotation matrix of the image acquisition device based on the distance between them and the shooting angle; and converting the homogeneous coordinates of each detection point into a second spatial position based on the translation vector and the rotation matrix.

[0078] In one embodiment, the device parameters include the focal length and optical center position of the image acquisition device. For each leg image, determining the homogeneous coordinates of each detection point based on the image position and device parameters includes: for each leg image, converting the image position of each detection point into homogeneous coordinates of each detection point based on the focal length and optical center position, wherein the homogeneous coordinates of each detection point are calculated according to the following formula (1):

[0079]

[0080] Among them, f x This refers to the focal length of the image acquisition device in the x-direction, f. y This refers to the focal length of the image acquisition device in the y-direction, c. x and c y [uv] refers to the position of the optical center of the image acquisition device, [uv] refers to the image position of each detection point in each leg image, [x] refers to the position of the optical center of the image acquisition device. c y c z c [] refers to the homogeneous coordinates of each detection point in each leg image. Homogeneous coordinates are spatial coordinates in the camera coordinate system with the origin of the first spatial position.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0086] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0087] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0089] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for assembling outriggers, characterized in that, The method includes: The outrigger image to be assembled is acquired by an image acquisition device, the outrigger including a first outrigger and a second outrigger, and the image acquisition device is connected to the end of a multi-axis robotic arm. For each outrigger image, determine the first spatial position of the image acquisition device, the distance between the image acquisition device and the outrigger to be assembled, and the shooting angle of the image acquisition device; For each leg image, the second spatial position of each detection point is determined based on the first spatial position, the distance between them, and the shooting angle; The deviation value between the first leg and the second leg is determined based on the second spatial position of all the detection points of the leg to be assembled; The center points of the first leg and the second leg are aligned in the length direction according to the deviation value for assembly.

2. The outrigger assembly method according to claim 1, characterized in that, The step of aligning the center points of the first leg and the second leg in the length direction according to the deviation value for assembly includes: The motion offset of the servo mechanism is determined based on the deviation value; The servo mechanism is controlled to move according to the motion offset, so as to keep the second leg stationary and control the center point of the first leg to move toward the center point of the second leg. If the deviation between the center point of the first leg and the center point of the second leg is less than or equal to a preset deviation threshold, it is determined that the center points of the first leg and the center points of the second leg are aligned in the length direction.

3. The outrigger assembly method according to claim 1, characterized in that, The method further includes: Before acquiring the image of the outrigger to be assembled through the image acquisition device, the outrigger model of the outrigger to be assembled is acquired. Determine the calibration procedure corresponding to the outrigger to be assembled based on the outrigger model; The movement trajectory and shooting angle of the image acquisition device are determined according to the calibration procedure; The image acquisition device is controlled to move according to the movement trajectory and to acquire images of the outrigger to be assembled at the shooting angle.

4. The outrigger assembly method according to claim 1, characterized in that, The step of determining the second spatial position of each detection point for each leg image, based on the first spatial position, the distance between points, and the second spatial position of each detection point, includes: Determine the image position of each detection point in each outrigger image and the device parameters of the image acquisition device; For each leg image, the homogeneous coordinates of each detection point are determined based on the image position of each detection point and the device parameters. The homogeneous coordinates refer to the spatial coordinates in the camera coordinate system with the origin of the first spatial position as the reference point. For each leg image, the homogeneous coordinates of each detection point are converted into the second spatial position based on the distance between them and the shooting angle.

5. The outrigger assembly method according to claim 4, characterized in that, The step of converting the homogeneous coordinates of each detection point into the second spatial position for each leg image based on the distance between them and the shooting angle includes: For each outrigger image, the translation vector and rotation matrix of the image acquisition device are determined based on the distance between them and the shooting angle; The homogeneous coordinates of each detection point are converted into the second spatial position based on the translation vector and the rotation matrix.

6. The outrigger assembly method according to claim 4, characterized in that, The device parameters include the focal length and optical center position of the image acquisition device. For each leg image, determining the homogeneous coordinates of each detection point based on the image position of each detection point and the device parameters includes: For each leg image, the image position of each detection point is converted into homogeneous coordinates of each detection point based on the focal length and the position of the optical center. The homogeneous coordinates of each detection point are calculated according to the following formula (1): Among them, f x This refers to the focal length of the image acquisition device in the x-direction, f. y This refers to the focal length of the image acquisition device in the y-direction, c. x and c y [uv] refers to the position of the optical center of the image acquisition device, [uv] refers to the image position of each detection point in each leg image, [x] refers to the position of the optical center of the image acquisition device. c y c z c ] refers to the homogeneous coordinates of each detection point in each leg image, where homogeneous coordinates are spatial coordinates in the camera coordinate system with the origin of the first spatial position.

7. A processor, characterized in that, It is configured to perform the outrigger assembly method according to any one of claims 1 to 6.

8. An engineering machinery, characterized in that, include: An image acquisition device is connected to the end effector of a multi-axis robotic arm, and the image acquisition device is used to acquire images of the outriggers to be assembled. A multi-axis robotic arm is used to control the movement trajectory and shooting angle of the image acquisition device; as well as The processor as described in claim 7.

9. The engineering machinery according to claim 8, characterized in that, Also includes: A servo mechanism is used to control the movement of the outrigger to be assembled.

10. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the leg assembly method according to any one of claims 1 to 6.

Images