Part assembly method, device and assembly system of device

Abstract

The invention provides a part assembling method, device and system of a device, the assembling system comprises an assembling robot, an assembling position, a camera and a moving mechanism, the assembling position is used for placing a to-be-assembled device, the camera is installed on the moving mechanism, and the moving mechanism is used for driving the camera to move in the length direction of the assembling position; the method comprises the following steps: controlling a moving mechanism to drive a camera to move along the length direction of an assembly position, and controlling the camera to photograph a device once at a preset time interval to obtain a plurality of images; determining an image containing a complete feature image in the plurality of images as a target image, wherein the feature image is an image of an assembly position of a part on the device; determining the assembly position of each part according to the target image; and the assembly robot is controlled to perform part assembly on the device according to the assembly position of each part, so that the problem of low part assembly efficiency of the device in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of parts assembly, and in particular to a device parts assembly method, device, computer-readable storage medium, computer program product and assembly system. Background Art

[0002] When assembling parts of large devices such as air-conditioning condensers, due to the assembly precision, vision and industrial robots need to be coordinated. The material volume is too large to be fully presented in the field of view of the industrial camera. When solving this type of problem, a common method is to move the large material left and right so that it is fully presented in the field of view of the industrial camera, take pictures to locate it, and then use an industrial robot to assemble it. This method requires a large space for the movement of large materials and has low photography efficiency. Summary of the invention

[0003] The main purpose of the present application is to provide a device parts assembly method, apparatus, computer-readable storage medium, computer program product and assembly system to at least solve the problem of low efficiency of device parts assembly in the prior art.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for assembling parts of a device is provided, the assembly system comprising an assembly robot, an assembly position, a camera and a moving mechanism, the assembly position is used to place the device to be assembled, the camera is installed on the moving mechanism, and the moving mechanism is used to drive the camera to move along the length direction of the assembly position, the method comprising: controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined interval to obtain multiple images; determining the image containing a complete feature image among the multiple images as a target image, the feature image being an image of the assembly position of a part on the device; determining the assembly position of each of the parts according to the target image; and controlling the assembly robot to assemble parts of the device according to the assembly position of each of the parts.

[0005] Optionally, the image containing a complete feature image among the multiple images is determined as the target image, including: performing feature extraction on the multiple images to obtain extracted feature images corresponding to each image; matching the extracted feature images corresponding to each image with the image of the installation position of the part to obtain multiple matching degrees; and determining the image whose matching degree is greater than a matching degree threshold as the target image.

[0006] Optionally, before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, the method also includes: establishing a workpiece coordinate system of the assembly robot so that the X-axis of the workpiece coordinate system is parallel to the length direction; when the camera is in the starting position, calibrating the pixel coordinate system of the camera and the workpiece coordinate system of the assembly robot to obtain a coordinate transformation matrix, and the starting position is the position of the camera before it moves.

[0007] Optionally, determining the assembly position of each of the parts according to the target image includes: determining corresponding pixel coordinates according to each of the target images, the pixel coordinates being the coordinates of the assembly position of the part in the pixel coordinate system; converting each of the pixel coordinates into intermediate coordinates according to the coordinate transformation matrix; calculating the movement distance of the camera corresponding to each of the images according to the number of each of the images, the predetermined duration and the movement speed of the camera; calculating the coordinates of the assembly position of each of the parts according to the movement distance of the camera corresponding to each of the images and the corresponding intermediate coordinates, the coordinates of the assembly position being the coordinates of the assembly position of the part in the workpiece coordinate system.

[0008] Optionally, the assembly system also includes a material tray, which is used to place the parts. The assembly robot is controlled to assemble parts of the device according to the assembly position, including: an assembly step, controlling the assembly robot to grab the parts on the material tray and move them to the corresponding assembly position for part assembly; repeating the assembly step at least once until all parts of the device are assembled.

[0009] Optionally, before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined interval to obtain multiple images, the method also includes: obtaining the distance between two adjacent assembly positions of the device in the length direction to obtain multiple adjacent distances; determining the minimum adjacent distance as the standard distance; setting the camera's moving speed and the predetermined time according to the standard distance, so that the distance moved by the camera within the predetermined time is less than or equal to half of the standard distance.

[0010] According to another aspect of the present application, a device for assembling parts of a device is provided, the assembly system comprising an assembly robot, an assembly position, a camera and a moving mechanism, the assembly position being used to place the device to be assembled, the camera being installed on the moving mechanism, the moving mechanism being used to drive the camera to move along the length direction of the assembly position, the device comprising: a first control unit, used to control the moving mechanism to drive the camera to move along the length direction of the assembly position, and to control the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images; a first determination unit, used to determine the image containing a complete feature image among the plurality of images as a target image, the feature image being an image of the assembly position of a part on the device; a second determination unit, used to determine the assembly position of each of the parts according to the target image; and a second control unit, used to control the assembly robot to assemble parts of the device according to the assembly position of each of the parts.

[0011] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0012] According to another aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, any one of the methods described above is implemented.

[0013] According to another aspect of the present application, an assembly system is provided, comprising: an assembly robot, an assembly position, a camera and a moving mechanism, one or more processors, a memory, and one or more programs, wherein the assembly position is used to place devices to be assembled, the camera is mounted on the moving mechanism, the moving mechanism is used to drive the camera to move along the length direction of the assembly position, the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods described.

[0014] By applying the technical solution of the present application, a parts assembly method for the above-mentioned device takes pictures of the device multiple times by moving a camera to obtain multiple images, and selects an image containing a complete feature image, i.e., a target image. The assembly position of each part can be determined according to the target image, thereby controlling the assembly robot to assemble the parts at the corresponding assembly position. This method can continuously capture multiple assembly positions at one time without moving the device for taking pictures, which greatly improves the shooting speed, thereby improving the assembly efficiency and solving the problem of low parts assembly efficiency of devices in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1A hardware structure block diagram of a mobile terminal for executing a method for assembling parts of a device provided in an embodiment of the present application is shown;

[0016] Figure 2 A schematic diagram of an assembly system provided according to an embodiment of the present application is shown;

[0017] Figure 3 A schematic flow chart of a method for assembling parts of a device provided in accordance with an embodiment of the present application is shown;

[0018] Figure 4 A schematic diagram of a camera capturing an image according to an embodiment of the present application is shown;

[0019] Figure 5 The figure shows the relationship between the coordinate system of an assembly robot and the coordinate system of a camera provided in an embodiment of the present application;

[0020] Figure 6 A schematic flow chart of a method for assembling parts of a device provided in accordance with an embodiment of the present application is shown;

[0021] Figure 7 A structural block diagram of a device parts assembly apparatus provided according to an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102. Processor; 104. Memory; 106. Transmission device; 108. Input and output device; 1. Application scenario platform; 2. Assembly robot; 3. Robot fixture; 4. Material tray; 5. Device; 6. Moving mechanism; 7. Camera; 8. Camera mounting plate. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, the efficiency of component assembly of devices in the prior art is low. To solve this technical problem, the embodiments of the present application provide a component assembly method, apparatus, computer-readable storage medium, computer program product and assembly system.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a device parts assembly method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the part assembly method of the device in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for assembling parts of a device running on a mobile terminal, a computer terminal or a similar computing device is provided. Figure 2 As shown, the assembly system includes an assembly robot 2, an assembly position, a camera 7 and a moving mechanism 6. The assembly position is used to place the device 5 to be assembled, and the camera 7 is installed on the moving mechanism 6. The moving mechanism 6 is used to drive the camera 7 to move along the length direction of the assembly position. The length direction is the direction from one side of the assembly position to the other side. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0032] Figure 3 FIG. 1 is a flow chart of a method for assembling parts of a device according to an embodiment of the present application. Figure 3 As shown, the method comprises the following steps:

[0033] Step S201, controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images;

[0034] Step S202, determining the image including the complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device;

[0035] Step S203, determining the assembly position of each of the above parts according to the above target image;

[0036] Step S204, controlling the assembly robot to assemble parts of the device according to the assembly positions of the parts.

[0037] The parts assembly method of the above-mentioned device takes pictures of the device multiple times by moving the camera to obtain multiple images, and selects the image containing the complete feature image, that is, the target image, so as to determine the assembly position of each part according to the target image, thereby controlling the assembly robot to assemble the parts at the corresponding assembly position. This method can continuously capture multiple assembly positions at one time without moving the device for taking pictures, which greatly improves the shooting speed, thereby improving the assembly efficiency and solving the problem of low parts assembly efficiency of devices in the prior art.

[0038] It should be noted that if Figure 2 As shown, the assembly system further includes an application scenario platform 1, a robot fixture 3 and a camera mounting plate 8. The camera is an industrial camera, the robot 2 is a four-axis robot, the moving mechanism 6 includes a linear module camera axis and a motor module thereof, a four-axis robot and a material tray 4 are arranged on the application scenario platform 1, and a linear module camera axis and a motor module thereof are installed above the device 5. The linear module camera axis and the motor module thereof communicate with the four-axis robot via a network cable, which serves as an external control camera axis module of the robot, and establishes the control of the four-axis robot over the linear module camera axis and the motor module thereof. The camera 7 is installed on the linear module camera axis and the motor module thereof through the camera mounting plate 8, so as to realize the rapid movement of the camera 7 and perform flying shooting. In addition, Figure 2 The enlarged images of parts A and B on the left are shown in Figure 2 on the right side.

[0039] In order to identify the matching degree, in an optional implementation, the above step S202 includes:

[0040] Step S2021, performing feature extraction on the plurality of images to obtain an extracted feature image corresponding to each of the images;

[0041] Step S2022, matching the extracted feature images corresponding to the images with the images of the installation positions of the parts to obtain a plurality of matching degrees;

[0042] Step S2023: determining the image whose matching degree is greater than a matching degree threshold as the target image.

[0043] In the above implementation mode, if Figure 4 As shown, the multiple images include image 1, image 2, image 3, ... and image n. The camera can judge and extract the feature image (feature point) in the image according to the template matching and the matching degree. Taking the complete concentric rings in the image as an example, the image without feature image is eliminated, such as Figure 4 Image 3 in , and the rest is the target image.

[0044] In order to realize the coordinate conversion, in an optional implementation manner, before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, the method further includes:

[0045] Step S301, establishing a workpiece coordinate system of the assembly robot so that the X-axis of the workpiece coordinate system is parallel to the length direction;

[0046] Step S302, when the camera is at a starting position, calibrate the pixel coordinate system of the camera and the workpiece coordinate system of the assembly robot to obtain a coordinate transformation matrix, wherein the starting position is the position of the camera before it moves.

[0047] In the above implementation mode, if Figure 5 As shown, since the base coordinate system {X1, Y1} of the four-axis robot and the coordinate system {X3} of the linear module camera axis and its motor module cannot ensure that the X-axis of the coordinate system coincides during mechanical assembly, which is not convenient for coordinate conversion. It is necessary to use the "two-point method" in the control system process package to calibrate the workpiece coordinate system WOBJ1 {X2, Y2} of the four-axis robot so that the X-axis of the robot workpiece coordinate system WOBJ1 {X2, Y2} coincides with the coordinate system {X3} of the linear module camera axis and its motor module. The robot subsequently runs in the WOBJ1 {X2, Y2} coordinate system in Cartesian coordinate mode to facilitate point coordinate conversion. The camera moves on the linear module camera axis and its motor module, and the camera's pixel coordinate system {X4, Y4} also moves on the coordinate system {X3} where the linear module camera axis and its motor module are located. However, the pixel coordinate system {X4, Y4} may have an angular offset with the coordinate system {X3} and the coordinate system {X2, Y2}. The industrial camera module is positioned at a certain point on the linear module camera axis and its motor module to make it in a static state. In the static state, the "nine-point method" hand-eye calibration is performed with the robot workpiece coordinate system WOBJ1 {X2, Y2}, and {X2, Y2} = cosθ{X4, Y4} is calculated based on the camera controller coordinate point position, thereby obtaining the transformation relationship between pixel coordinates and mechanical coordinates, that is, the coordinate transformation matrix.

[0048] In order to perform coordinate conversion, in an optional implementation, the above step S203 includes:

[0049] Step S2031, determining corresponding pixel coordinates according to each of the target images, the pixel coordinates being coordinates of the assembly position of the part in the pixel coordinate system;

[0050] Step S2032, converting each of the above pixel coordinates into intermediate coordinates according to the above coordinate transformation matrix;

[0051] Step S2033, calculating the moving distance of the camera corresponding to each of the images according to the number of each of the images, the predetermined time duration and the moving speed of the camera;

[0052] Step S2034, calculating the coordinates of the assembly position of each of the above parts according to the movement distance of the above camera corresponding to each of the above images and the corresponding intermediate coordinates, and the coordinates of the above assembly position are the coordinates of the assembly position of the above part in the above workpiece coordinate system.

[0053] In the above implementation, each image captured by the camera can determine the pixel coordinates (x4, y4) of the assembly position, and then use the conversion relationship between the pixel coordinates and the mechanical coordinates {X2, Y2} = cosθ{X4, Y4} to obtain the mechanical coordinates (x2, y2) of the assembly position. However, since the pixel coordinates move in a straight line on the coordinate system {X3}, at the nth image acquisition point, the mechanical coordinates need to be compensated for the coordinate value in the X-axis direction. According to the principle of linear motion, the compensation value L = nsv, and the mechanical coordinates of the assembly position are (x2+L, y2).

[0054] In order to improve the assembly efficiency, in an optional implementation manner, the assembly system further includes a material tray 4, and the material tray 4 is used to place the parts. The step S204 includes:

[0055] Step S2041, an assembly step, controlling the assembly robot to grab the parts on the material tray and move them to the corresponding assembly positions to assemble the parts;

[0056] Step S2042, repeat the above assembly steps at least once until all parts of the above device are assembled.

[0057] In the above implementation, all parts of the device can be assembled in sequence by collecting and positioning multi-position feature images at one time.

[0058] In order to avoid missing an assembly position, in an optional implementation manner, before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position and controlling the camera to take a picture of the device once at a predetermined interval to obtain a plurality of images, the method further includes:

[0059] Step S401, obtaining the distance between two adjacent assembly positions of the device in the length direction to obtain a plurality of adjacent distances;

[0060] Step S402, determining the minimum adjacent distance as the standard distance;

[0061] Step S403: setting the moving speed of the camera and the predetermined time period according to the standard distance, so that the distance moved by the camera within the predetermined time period is less than or equal to half of the standard distance.

[0062] In the above embodiment, the distance that the camera moves within the above predetermined time period, that is, the moving distance for one photograph should not be too large. Taking the minimum adjacent distance as a reference, the moving speed of the camera and the above predetermined time period are increased, and the moving distance for one photograph is controlled to be less than or equal to half of the above standard distance to avoid missing the assembly position. It should not be too small to avoid too many images and a large screening workload.

[0063] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the parts assembly method of the device of the present application will be described in detail below in conjunction with specific embodiments.

[0064] This embodiment relates to a specific device parts assembly method, such as Figure 6 As shown, the following steps are included:

[0065] A four-axis robot module, a material tray and materials are arranged on the application scenario platform. A linear module camera axis and its motor module are installed above the large material to be photographed. The linear module camera axis and its motor module communicate with the four-axis robot module via a network cable, which serves as the robot's external control camera axis module, and establishes control of the linear module camera axis and its motor module by the four-axis robot module. The industrial camera module is installed on the linear module camera axis and its motor module through the industrial camera mounting plate, so as to realize the rapid movement of the industrial camera module and perform flying shooting.

[0066] The external camera axis control system process package needs to use the network port network communication module to establish network communication between the four-axis robot module, the linear module camera axis and its motor module, and the industrial camera module. After establishing the communication, the relevant motion parameters (motor speed, encoder bit number, etc.) of the linear module camera axis and its motor module are edited in the parameter module of the control system process package in the present invention, so as to achieve real-time control of the motion state of the linear module camera axis and its motor module by the four-axis robot module.

[0067] Since the base coordinate system {X1, Y1} of the four-axis robot module and the coordinate system {X3} of the linear module camera axis and its motor module cannot be guaranteed to coincide with each other in the mechanical assembly, which is not convenient for coordinate conversion. It is necessary to use the "two-point method" in the control system process package to calibrate the workpiece coordinate system WOBJ1 {X2, Y2} of the four-axis robot module so that the X-axis of the robot workpiece coordinate system WOBJ1 {X2, Y2} coincides with the coordinate system {X3} of the linear module camera axis and its motor module. The robot subsequently runs in the WOBJ1 {X2, Y2} coordinate system in Cartesian coordinate mode to facilitate point coordinate conversion.

[0068] The industrial camera module moves on the linear module camera axis and its motor module. The pixel coordinate system {X4, Y4} of the industrial camera module also moves on the coordinate system {X3} where the linear module camera axis and its motor module are located. However, the pixel coordinate system {X4, Y4} may have an angular offset with the coordinate system {X3} and the coordinate system {X2, Y2}. The industrial camera module is positioned at a certain point on the linear module camera axis and its motor module so that it is in a static state. In the static state, the "nine-point method" hand-eye calibration is performed with the robot workpiece coordinate system WOBJ1 {X2, Y2}, and {X2, Y2} = cosθ{X4, Y4} is calculated based on the coordinate point position of the industrial camera module controller, thereby obtaining the conversion relationship between pixel coordinates and mechanical coordinates.

[0069] After the coordinate transformation relationship is established, the four-axis robot module can be edited in the parameter module included in the control system process package to control the speed v of the linear module camera axis and its motor module, and to take photos and collect pictures every s seconds during the movement. After setting the above two parameter modules, the linear module camera axis and its motor module can be controlled through network communication. At the same time, the industrial camera module can be controlled to collect images during the movement. The industrial camera module moves and takes pictures under the control of the robot to obtain n images. The camera system numbers the graphics. The camera can judge and extract feature points in the image based on template matching (taking the complete concentric circles in the image as an example), and eliminate images without feature points. Since the camera is taken under the control of the robot, the robot system stores the time of taking pictures, such as the time of taking the nth image is ns seconds. The pixel coordinates (x4, y4) of the feature point can be determined for each image captured by the camera, and then the conversion relationship between pixel coordinates and mechanical coordinates {X2, Y2} = cosθ{X4, Y4} is used to obtain the mechanical coordinates (x2, y2) of the feature point. However, since the pixel coordinates move linearly on the coordinate system {X3}, at the nth image capture point, the mechanical coordinates need to be compensated for the coordinate value in the X-axis direction. According to the principle of linear motion, the compensation value L = nsv, and the mechanical coordinates of the feature point are (x2 + L, y2). The industrial camera module controller sends the pixel coordinates and numbers of all images with feature points at the same time, and the robot control system uses a multi-threaded algorithm to calculate all the mechanical coordinates of the feature point positions collected at one time, which is the robot execution point. The four-axis robot module uses the fixture module to clamp the material and then continuously and sequentially remove the feature points for assembly, thereby realizing the operation of the external camera axis control system process package.

[0070] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0071] The present application embodiment also provides a kind of parts assembly device of device, it should be noted that the parts assembly device of the device of the present application embodiment can be used to execute the parts assembly method for device provided by the present application embodiment. The device is used to implement the above-mentioned embodiment and preferred implementation mode, and the description has been made and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0072] The following is an introduction to the parts assembly device of the device provided in an embodiment of the present application. The assembly system includes an assembly robot, an assembly position, a camera and a moving mechanism. The assembly position is used to place the device to be assembled. The camera is installed on the moving mechanism. The moving mechanism is used to drive the camera to move along the length direction of the assembly position. The length direction is the direction from one side of the assembly position to the other side.

[0073] Figure 7 is a structural block diagram of a device for assembling parts according to an embodiment of the present application. Figure 7 As shown, the device comprises:

[0074] A first control unit 10 is used to control the moving mechanism to drive the camera to move along the length direction of the assembly position, and control the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images;

[0075] A first determining unit 20 is used to determine the image including the complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device;

[0076] A second determining unit 30 is used to determine the assembly position of each of the above parts according to the above target image;

[0077] The second control unit 40 is used to control the assembly robot to assemble parts of the device according to the assembly positions of the parts.

[0078] In the parts assembly device of the above-mentioned device, the device is photographed multiple times by moving the camera to obtain multiple images, and the image containing the complete feature image, i.e., the target image, is selected. The assembly position of each part can be determined according to the target image, thereby controlling the assembly robot to assemble the parts in the corresponding assembly position. This method can continuously capture multiple assembly positions at one time without moving the device for photographing, which greatly improves the shooting speed, thereby improving the assembly efficiency and solving the problem of low parts assembly efficiency of devices in the prior art.

[0079] It should be noted that if Figure 2As shown, the assembly system further includes an application scenario platform 1, a robot fixture 3 and a camera mounting plate 8. The camera is an industrial camera, the robot 2 is a four-axis robot, the moving mechanism 6 includes a linear module camera axis and a motor module thereof, a four-axis robot and a material tray 4 are arranged on the application scenario platform 1, and a linear module camera axis and a motor module thereof are installed above the device 5. The linear module camera axis and the motor module thereof communicate with the four-axis robot via a network cable, which serves as an external control camera axis module of the robot, and establishes the control of the four-axis robot over the linear module camera axis and the motor module thereof. The camera 7 is installed on the linear module camera axis and the motor module thereof through the camera mounting plate 8, so as to realize the rapid movement of the camera 7 and perform flying shooting. In addition, Figure 2 The enlarged images of parts A and B on the left are shown in Figure 2 on the right side.

[0080] In order to identify the matching degree, in an optional implementation manner, the first determining unit includes:

[0081] An extraction module, used to extract features from the plurality of images to obtain extracted feature images corresponding to the respective images;

[0082] A matching module, used for matching the extracted feature images corresponding to the images with the images of the installation positions of the parts to obtain a plurality of matching degrees;

[0083] The first determination module is used to determine the image whose matching degree is greater than a matching degree threshold as the target image.

[0084] In the above implementation mode, if Figure 4 As shown, the multiple images include image 1, image 2, image 3, ... and image n. The camera can judge and extract the feature image (feature point) in the image according to the template matching and the matching degree. Taking the complete concentric rings in the image as an example, the image without feature image is eliminated, such as Figure 4 Image 3 in , and the rest is the target image.

[0085] In order to realize coordinate conversion, in an optional implementation manner, the above-mentioned device further includes:

[0086] An establishing unit, used for establishing a workpiece coordinate system of the assembly robot before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, so that the X axis of the workpiece coordinate system is parallel to the length direction;

[0087] The calibration unit is used to calibrate the pixel coordinate system of the camera and the workpiece coordinate system of the assembly robot to obtain a coordinate transformation matrix when the camera is at a starting position, and the starting position is the position of the camera before it moves.

[0088] In the above implementation mode, if Figure 5 As shown, since the base coordinate system {X1, Y1} of the four-axis robot and the coordinate system {X3} of the linear module camera axis and its motor module cannot ensure that the X-axis of the coordinate system coincides during mechanical assembly, which is not convenient for coordinate conversion. It is necessary to use the "two-point method" in the control system process package to calibrate the workpiece coordinate system WOBJ1 {X2, Y2} of the four-axis robot so that the X-axis of the robot workpiece coordinate system WOBJ1 {X2, Y2} coincides with the coordinate system {X3} of the linear module camera axis and its motor module. The robot subsequently runs in the WOBJ1 {X2, Y2} coordinate system in Cartesian coordinate mode to facilitate point coordinate conversion. The camera moves on the linear module camera axis and its motor module, and the camera's pixel coordinate system {X4, Y4} also moves on the coordinate system {X3} where the linear module camera axis and its motor module are located. However, the pixel coordinate system {X4, Y4} may have an angular offset with the coordinate system {X3} and the coordinate system {X2, Y2}. The industrial camera module is positioned at a certain point on the linear module camera axis and its motor module to make it in a static state. In the static state, the "nine-point method" hand-eye calibration is performed with the robot workpiece coordinate system WOBJ1 {X2, Y2}, and {X2, Y2} = cosθ{X4, Y4} is calculated based on the camera controller coordinate point position, thereby obtaining the transformation relationship between pixel coordinates and mechanical coordinates, that is, the coordinate transformation matrix.

[0089] In order to perform coordinate conversion, in an optional implementation manner, the second determining unit includes:

[0090] A second determination module is used to determine corresponding pixel coordinates according to each of the target images, where the pixel coordinates are coordinates of the assembly position of the part in the pixel coordinate system;

[0091] A conversion module, used for converting each of the above pixel coordinates into an intermediate coordinate according to the above coordinate transformation matrix;

[0092] A first calculation module, used for calculating the moving distance of the camera corresponding to each of the images according to the number of each of the images, the predetermined time length and the moving speed of the camera;

[0093] The second calculation module is used to calculate the coordinates of the above-mentioned assembly position of each of the above-mentioned parts according to the movement distance of the above-mentioned camera corresponding to each of the above-mentioned images and the corresponding above-mentioned intermediate coordinates. The coordinates of the above-mentioned assembly position are the coordinates of the above-mentioned assembly position of the above-mentioned part in the above-mentioned workpiece coordinate system.

[0094] In the above implementation, each image captured by the camera can determine the pixel coordinates (x4, y4) of the assembly position, and then use the conversion relationship between the pixel coordinates and the mechanical coordinates {X2, Y2} = cosθ{X4, Y4} to obtain the mechanical coordinates (x2, y2) of the assembly position. However, since the pixel coordinates move in a straight line on the coordinate system {X3}, at the nth image acquisition point, the mechanical coordinates need to be compensated for the coordinate value in the X-axis direction. According to the principle of linear motion, the compensation value L = nsv, and the mechanical coordinates of the assembly position are (x2+L, y2).

[0095] In order to improve the assembly efficiency, in an optional implementation manner, the assembly system further includes a material tray 4, the material tray 4 is used to place the parts, and the second control unit includes:

[0096] An assembly module, used to execute assembly steps, control the assembly robot to grab the parts on the material tray and move them to the corresponding assembly positions for part assembly;

[0097] The repeating module is used to repeat the above assembly steps at least once until all parts of the above device are assembled.

[0098] In the above implementation, all parts of the device can be assembled in sequence by collecting and positioning multi-position feature images at one time.

[0099] In order to avoid missing the assembly position, in an optional implementation manner, the above device further includes:

[0100] an acquisition unit, for acquiring a distance between two adjacent assembly positions of the device in the length direction before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position and controlling the camera to take a picture of the device once at a predetermined interval to obtain a plurality of images, so as to obtain a plurality of adjacent distances;

[0101] A third determining unit, configured to determine the minimum adjacent distance as a standard distance;

[0102] The setting unit is used to set the moving speed of the camera and the predetermined time according to the standard distance, so that the distance moved by the camera within the predetermined time is less than or equal to half of the standard distance.

[0103] In the above embodiment, the distance that the camera moves within the above predetermined time period, that is, the moving distance for one photograph should not be too large. Taking the minimum adjacent distance as a reference, the moving speed of the camera and the above predetermined time period are increased, and the moving distance for one photograph is controlled to be less than or equal to half of the above standard distance to avoid missing the assembly position. It should not be too small to avoid too many images and a large screening workload.

[0104] The parts assembly device of the device includes a processor and a memory. The first control unit, the first determination unit, the second determination unit, and the second control unit are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0105] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be provided, and the problem of low efficiency of component assembly of the device in the prior art can be solved by adjusting kernel parameters.

[0106] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0107] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the parts assembly method of the device.

[0108] Specifically, the parts assembly method of the device includes:

[0109] Step S201, controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images;

[0110] Step S202, determining the image including the complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device;

[0111] Step S203, determining the assembly position of each of the above parts according to the above target image;

[0112] Step S204, controlling the assembly robot to assemble parts of the device according to the assembly positions of the parts.

[0113] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the parts assembly method of the device is executed when the program is run.

[0114] Specifically, the parts assembly method of the device includes:

[0115] Step S201, controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images;

[0116] Step S202, determining the image including the complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device;

[0117] Step S203, determining the assembly position of each of the above parts according to the above target image;

[0118] Step S204, controlling the assembly robot to assemble parts of the device according to the assembly positions of the parts.

[0119] An embodiment of the present invention provides an assembly system, which includes an assembly robot, an assembly position, a camera, a moving mechanism, a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are implemented:

[0120] Step S201, controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images;

[0121] Step S202, determining the image including the complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device;

[0122] Step S203, determining the assembly position of each of the above parts according to the above target image;

[0123] Step S204, controlling the assembly robot to assemble parts of the device according to the assembly positions of the parts.

[0124] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0125] Step S201, controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images;

[0126] Step S202, determining the image including the complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device;

[0127] Step S203, determining the assembly position of each of the above parts according to the above target image;

[0128] Step S204, controlling the assembly robot to assemble parts of the device according to the assembly positions of the parts.

[0129] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0136] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0138] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0139] 1) The parts assembly method of the device of the present application obtains multiple images by moving the camera to take pictures of the device multiple times, and selects the image containing the complete feature image, that is, the target image. The assembly position of each part can be determined according to the target image, so as to control the assembly robot to assemble the parts in the corresponding assembly position. This method can continuously capture multiple assembly positions at one time without moving the device for taking pictures, which greatly improves the shooting speed, thereby improving the assembly efficiency and solving the problem of low parts assembly efficiency of devices in the prior art.

[0140] 2) In the parts assembly device of the device of the present application, the device is photographed multiple times by moving the camera to obtain multiple images, and the image containing the complete feature image, i.e., the target image, is selected. The assembly position of each part can be determined according to the target image, thereby controlling the assembly robot to assemble the parts in the corresponding assembly position. This method can continuously capture multiple assembly positions at one time without moving the device for photographing, which greatly improves the shooting speed, thereby improving the assembly efficiency and solving the problem of low parts assembly efficiency of devices in the prior art.

[0141] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for assembling parts of a device, characterized in that: The assembly system comprises an assembly robot, an assembly position, a camera and a moving mechanism, wherein the assembly position is used to place the device to be assembled, the camera is mounted on the moving mechanism, and the moving mechanism is used to drive the camera to move along the length direction of the assembly position. The method comprises: Controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, and controlling the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images; Determine the image containing a complete characteristic image among the plurality of images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device; Determining the assembly position of each of the parts according to the target image; The assembly robot is controlled to assemble parts of the device according to the assembly position of each part.

2. The method according to claim 1, characterized in that Determining the image containing the complete feature image among the plurality of images as the target image comprises: Performing feature extraction on the plurality of images to obtain an extracted feature image corresponding to each of the images; Matching the extracted feature image corresponding to each of the images with the image of the installation position of the part to obtain a plurality of matching degrees; The image whose matching degree is greater than a matching degree threshold is determined as the target image.

3. The method according to claim 1, characterized in that Before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position, the method further includes: Establishing a workpiece coordinate system of the assembly robot so that an X-axis of the workpiece coordinate system is parallel to the length direction; When the camera is at the starting position, the pixel coordinate system of the camera and the workpiece coordinate system of the assembly robot are calibrated to obtain a coordinate transformation matrix, and the starting position is the position of the camera before it moves.

4. The method according to claim 3, characterized in that Determining the assembly position of each of the parts according to the target image includes: Determine corresponding pixel coordinates according to each of the target images, wherein the pixel coordinates are coordinates of the assembly position of the part in the pixel coordinate system; Convert each of the pixel coordinates into an intermediate coordinate according to the coordinate transformation matrix; Calculating the moving distance of the camera corresponding to each of the images according to the number of each of the images, the predetermined time duration and the moving speed of the camera; The coordinates of the assembly position of each part are calculated according to the moving distance of the camera corresponding to each image and the corresponding intermediate coordinates, and the coordinates of the assembly position are the coordinates of the assembly position of the part in the workpiece coordinate system.

5. The method according to claim 1, characterized in that The assembly system further includes a material tray, the material tray is used to place the parts, and the assembly robot is controlled to assemble parts of the device according to the assembly position, including: An assembly step, controlling the assembly robot to grab the parts on the material tray and move them to the corresponding assembly positions for part assembly; The assembling steps are repeated at least once until all parts of the device are assembled.

6. The method according to any one of claims 1 to 5, characterized in that Before controlling the moving mechanism to drive the camera to move along the length direction of the assembly position and controlling the camera to take a picture of the device once at a predetermined interval to obtain a plurality of images, the method further includes: Obtaining the distance between two adjacent assembly positions of the device in the length direction to obtain a plurality of adjacent distances; Determine the minimum adjacent distance as the standard distance; The moving speed of the camera and the predetermined time period are set according to the standard distance, so that the distance moved by the camera within the predetermined time period is less than or equal to half of the standard distance.

7. A device for assembling parts of a device, characterized in that: The assembly system includes an assembly robot, an assembly position, a camera and a moving mechanism, wherein the assembly position is used to place the device to be assembled, the camera is mounted on the moving mechanism, and the moving mechanism is used to drive the camera to move along the length direction of the assembly position. The device includes: A first control unit is used to control the moving mechanism to drive the camera to move along the length direction of the assembly position, and control the camera to take a picture of the device once at a predetermined time interval to obtain a plurality of images; A first determining unit is used to determine the image containing a complete characteristic image among the multiple images as a target image, wherein the characteristic image is an image of an assembly position of a part on the device; A second determining unit, used to determine the assembly position of each of the parts according to the target image; The second control unit is used to control the assembly robot to assemble parts of the device according to the assembly position of each part.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. An assembly system, characterized in that: include: An assembly robot, an assembly position, a camera and a moving mechanism, one or more processors, a memory, and one or more programs, wherein the assembly position is used to place devices to be assembled, the camera is mounted on the moving mechanism, and the moving mechanism is used to drive the camera to move along the length direction of the assembly position, the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of claims 1 to 6.

Images